Back to Blog
Company Updates

Building Fyllyo: My Journey from Idea to Launch

Behind the scenes look at building an AI-powered form filling solution. The challenges we faced, lessons we learned, and what's coming next.

November 28, 2024
10 min read

Every startup has an origin story. Mine started with a simple frustration: spending 3 hours filling out a visa application form.

The "Aha!" Moment

I have felt the same frustration every time I fill out a visa application form. tried different extentions, nothing actually worked well. As a software developer, I always symphatized, because there is just som much variation in how forms are built. With recent AI advancements things changed, and I think we can finally solve this problem. You have data, you should be required to type it 1000s of times. With AI at our disposal this must change. So I built Fyllyo for this very mission.

The Lightbulb Moment

"What if a tool could remember all this information and fill it out for me? Not just basic auto-fill, but actually understand the context and requirements of different forms."

The Early Days: Research and Reality Checks

As a developer I leaned on building MVP before any validation. I did do research on existing solutions and found out there aree different tools for slightly different problems. But mostly built with pre llm era mindset.

😤 Tools in market

  • • PDF form fillers based on old AI models
  • • Chrome extensions that only work on certain sites
  • • Manual form fillers that require you type once and they can fill exact same data again
  • • Agentic browsers that automate a task but would not store your data
  • • Browser automation where you dont have visiblity or control

💡 The Opportunities

  • • Need a tool that can fill any form
  • • I can quickly see what was filled and validate before submitting
  • • I can store my data and use it across different forms
  • • Smart enough to extract context from the form

Logic sounded solid and I am true believer of value creation, when you have experienced those pain points.

Technical Challenges: The Hard Problems

Challenge #1: Form Recognition

Every website builds forms differently. Some use standard HTML, others use complex JavaScript frameworks. Government sites often have legacy systems with unusual structures. AI needed to understand them all.

My Solution: Its simple if the website is using standard html input fields. LLMs can easly understand hidden fields, their relationships and context, decide which fields are mirrored. But many modern websites use custom ui components. Thats where i annovated and added llm powered pattern matching based on html tag properties and attributes. This allows fyllyo to dynamically understand what could be form fields and fill them.

Challenge #2: Context Understanding

A "name" field on a job application needs different information than a "name" field on a visa form. The AI needed to understand context, not just field labels. Which is again easy problem to solve with llm. Tricky bit is many websites donot have proper labels or use random hierarchies of headings and sections that may contain relevan information. A form may ask for your mother's info but all fields are general name, address, phone number, etc. And only the section or page heading has some hint that it is mother's info.

My Solution: We developed a context-aware AI that analyzes the entire page, understands the form's purpose, and adapts its responses accordingly. This was our biggest breakthrough. I solved this in fyllyo by creating a path for each form field. Traversing through text nodes starting from the input to the top of the page

Challenge #3: Privacy and Security

Users were trusting us with their most sensitive information: SSNs, passport numbers, financial data. We couldn't just build a great product—we had to build a secure one.

🔒 Security First

We made a crucial decision early: all sensitive data stays on the user's device. Being a security engineer, I knew this was a non-negotiable requirement. And built the MVP with all security features in place.

What's Next: The road ahead

I need to do a lot of work to reach out to users who can benefit from this. I will keep updating the blog with my progress and updates.

Thanks for reading this.

🚀 Join Our Journey

I am always looking for feedback, feature requests, and stories about how Fyllyo has helped you.

The future of form filling is intelligent, secure, and user-centric. We're excited to continue building that future with you.

Related Articles