Screenshot to Code?

Part 1 (Jan/2024)

Came across and tried Screenshot to Code a while ago.

It accepts an image as the input and, with the help of AI, generates source code that resembles the input.

After setting up the environment, start the frontend and a screen like the following shows:

I tried to feed a screenshot of Instagram login screen into the application. Here is what I got:

It's OK, but not so impressive. It will be useful for learning new tools, for example, it is pretty good as learning material for Tailwind.


Part 2 (June/2024)

Six months later, I revisited the same application.

It has a lot of improvement. It can even support video as an input, as well as finetuning on top of your generated result.

I use GPT-4o this time, and let's try a page with more content.

Here is the original screenshot of the GitHub co-pilot product page:

And, the AI-generated version:

And, the code:

<html lang="en">
 <head>
  <meta charset="utf-8"/>
  <meta content="width=device-width, initial-scale=1.0" name="viewport"/>
  <title>
   GitHub Copilot
  </title>
  <script src="https://cdn.tailwindcss.com">
  </script>
  <link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css" rel="stylesheet"/>
  <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;700&amp;display=swap" rel="stylesheet"/>
  <style>
   body {
            font-family: 'Inter', sans-serif;
        }
  </style>
 </head>
 <body class="bg-gradient-to-b from-black to-blue-900 text-white">
  <!-- Top Banner -->
  <div class="bg-purple-600 text-center py-2 text-sm">
   Join us for the 10th anniversary of GitHub Universe. Get tickets now for 35% off, only until July 8.
  </div>
  <!-- Navbar -->
  <nav class="bg-black py-4">
   <div class="container mx-auto flex items-center justify-between px-4">
    <div class="flex items-center space-x-4">
     <img alt="GitHub logo" class="h-8 w-8" height="40" src="https://oaidalleapiprodscus.blob.core.windows.net/private/org-cRVBdRBrtezhsZz1Nrf6eYR3/user-FERV71JOjDv6zSVyrzALWChB/img-Pa6Mt93TPPp1lTPjfL9RhUyW.png?st=2024-06-23T07%3A53%3A06Z&amp;se=2024-06-23T09%3A53%3A06Z&amp;sp=r&amp;sv=2023-11-03&amp;sr=b&amp;rscd=inline&amp;rsct=image/png&amp;skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&amp;sktid=a48cca56-e6da-484e-a814-9c849652bcb3&amp;skt=2024-06-23T00%3A04%3A22Z&amp;ske=2024-06-24T00%3A04%3A22Z&amp;sks=b&amp;skv=2023-11-03&amp;sig=mC06M6q2sY6BHts69ipcu0x7TElzDrjUT47tn9/nLR0%3D" width="40"/>
     <div class="text-white flex space-x-4">
      <div class="relative group">
       <button class="focus:outline-none">
        Product
        <i class="fas fa-chevron-down">
        </i>
       </button>
       <!-- Dropdown content can be added here -->
      </div>
      <div class="relative group">
       <button class="focus:outline-none">
        Solutions
        <i class="fas fa-chevron-down">
        </i>
       </button>
       <!-- Dropdown content can be added here -->
      </div>
      <div class="relative group">
       <button class="focus:outline-none">
        Open Source
        <i class="fas fa-chevron-down">
        </i>
       </button>
       <!-- Dropdown content can be added here -->
      </div>
      <div class="relative group">
       <button class="focus:outline-none">
        Enterprise
        <i class="fas fa-chevron-down">
        </i>
       </button>
       <!-- Dropdown content can be added here -->
      </div>
      <a class="focus:outline-none" href="#">
       Pricing
      </a>
     </div>
    </div>
    <div class="flex items-center space-x-4">
     <input class="bg-gray-800 text-white px-4 py-2 rounded-md focus:outline-none" placeholder="Search or jump to..." type="text"/>
     <a class="text-white" href="#">
      Sign in
     </a>
     <a class="bg-white text-black px-4 py-2 rounded-md" href="#">
      Sign up
     </a>
    </div>
   </div>
  </nav>
  <!-- Sub Navbar -->
  <div class="bg-black py-2">
   <div class="container mx-auto flex items-center justify-between px-4 text-gray-400">
    <a class="text-white font-bold" href="#">
     Features
    </a>
    <a href="#">
     Actions
    </a>
    <a href="#">
     Packages
    </a>
    <a href="#">
     Security
    </a>
    <a href="#">
     Codespaces
    </a>
    <a class="text-white" href="#">
     Copilot
    </a>
    <a href="#">
     Code review
    </a>
    <a href="#">
     Search
    </a>
    <a href="#">
     Issues
    </a>
    <a href="#">
     Discussions
    </a>
   </div>
  </div>
  <!-- Main Content -->
  <div class="container mx-auto text-center py-20 px-4">
   <div class="inline-block bg-gray-800 text-white px-4 py-2 rounded-full mb-4">
    <i class="fas fa-robot">
    </i>
    Copilot Enterprise now available
   </div>
   <h1 class="text-5xl font-bold mb-6">
    The world’s most widely adopted AI developer tool.
   </h1>
   <a class="bg-white text-black px-6 py-3 rounded-md" href="#">
    Get started with Copilot
    <i class="fas fa-arrow-right">
    </i>
   </a>
  </div>
 </body>
</html>

How much does it cost?

After running 3 different screenshots, the OpenAI dashboard shows $0.35 ($0.28 from the image model, and $0.07 from GPT-4o). So that is about $0.12/run.


AI Summary
gpt-4o-2024-05-13 2024-07-16 00:40:56
The blog explores the evolution of the AI tool "Screenshot to Code," which converts images into source code. Initially tested in January 2024, the tool showed moderate results but, by June 2024, it improved significantly, supporting video input and fine-tuning capabilities. The cost per usage is approximately $0.12.
Chrome On-device AI 2024-07-20 06:01:12

Share Article