Ask HN: If I wanted to make a web browser, where would I start?
55 comments
Is it still even doable for a single person to write a new rendering engine from scratch? I'm getting the impression that right now it's all about Webkit, Gecko and IE and given how complex things got, I imagine that it would cost an enormous sum of money to write something that, say, can display top 10 Alexa web pages. Things just got too complex.
Perhaps it would be better to have some effort in documenting Webkit code so that more hackers could actually read it and hack on it?
Perhaps it would be better to have some effort in documenting Webkit code so that more hackers could actually read it and hack on it?
It shouldn't be too much for a single person to take on if they're truly dedicated to the task. A small team is still probably more ideal, but the browser isn't overly complex, it simply has a lot of capability that it can pull off with its feature set. All of a browsers moving parts are designed to be quite generic, which allows quite complex interactions to arise out of a small handful of simpler components.
Things used to be a lot harder 5 years ago, but the renewed focus on standards means that many smaller browsers can focus on just that, and ignore a lot of the weird quirks found in major browsers safely. (The death of Internet Explorer is finally bringing about the necessary changes in websites that have clung to its quirks mode so blindly for so many years.)
I think starting with a manually written parser is a necessary first step if you're writing a web browser. It's tempting to want to pull in Webkit or V8 or Gecko or Chakra, but if you're writing a web browser, you need to understand what these tools are actually doing for you before you use them. Writing your own versions, even if that's time consuming, will teach you why these other frameworks have become so popular, and educate you about their shortcomings so you can hopefully code around those too. If you do it write, and your codebase is appropriately modular, it should be straightforward to modify the project down the road to use a different parser, or even an entirely different DOM tree implementation.
Things used to be a lot harder 5 years ago, but the renewed focus on standards means that many smaller browsers can focus on just that, and ignore a lot of the weird quirks found in major browsers safely. (The death of Internet Explorer is finally bringing about the necessary changes in websites that have clung to its quirks mode so blindly for so many years.)
I think starting with a manually written parser is a necessary first step if you're writing a web browser. It's tempting to want to pull in Webkit or V8 or Gecko or Chakra, but if you're writing a web browser, you need to understand what these tools are actually doing for you before you use them. Writing your own versions, even if that's time consuming, will teach you why these other frameworks have become so popular, and educate you about their shortcomings so you can hopefully code around those too. If you do it write, and your codebase is appropriately modular, it should be straightforward to modify the project down the road to use a different parser, or even an entirely different DOM tree implementation.
The above list is a good start. I'd add: 2a) make clicking on links work, and 3a) display images. Those two features will make it much more fun.
"How to make a browser" is something I've thought about for a while, ever since someone suggested it as an interview question. (I think it would be an awful question.) It really depends on what you're trying to do. If you want to understand how browsers work and build a toy browser, follow jbb555's list. If you want to display web pages, use WebKit. If you want to build a real, commercial-scale browser, I recommend a large team, since there are a lot of components, each of which is insanely complex. (Everything from CSS support to security to plugins to JavaScript to bookmarks to a debugger to all the new network protocols.)
One more thing to add to jbb555's list: 0) "telnet news.ycombinator.com 80, GET /" - playing around from the command line can give you a good introduction to what really happens when you access a web page. (Edit: HN returns an error page for a plain GET. google.com returns insane code. apple.com looks like a better place to start.)
"How to make a browser" is something I've thought about for a while, ever since someone suggested it as an interview question. (I think it would be an awful question.) It really depends on what you're trying to do. If you want to understand how browsers work and build a toy browser, follow jbb555's list. If you want to display web pages, use WebKit. If you want to build a real, commercial-scale browser, I recommend a large team, since there are a lot of components, each of which is insanely complex. (Everything from CSS support to security to plugins to JavaScript to bookmarks to a debugger to all the new network protocols.)
One more thing to add to jbb555's list: 0) "telnet news.ycombinator.com 80, GET /" - playing around from the command line can give you a good introduction to what really happens when you access a web page. (Edit: HN returns an error page for a plain GET. google.com returns insane code. apple.com looks like a better place to start.)
Yeah, it wasn't intended to be a well thought out list.
It was meant to be "just start and iterate" but a little more specific :)
Pretty much this.
- Write the networking code to retrieve data from servers. I'd start with GET requests to keep it simple. This part is not that bad and its actually pretty fun. Doing it in Python is very simple and straightforward.
- Write browser GUI because its a native application. This depends on what language you use for the task. If Python, you can get away with tkinter (flame suit on) for a very simple prototype. I believe tkinter is single threaded and that may turn out to be an issue (don't remember).
- Write a simple rendering engine to display the text. No need to do this inside of the GUI code because the output can be read through the console. You are pretty much reading the raw html and extracting the bits you want.
- Plug in the rendering engine into a GUI widget. Since this is a (assumed) read-only browser, you can get away with a simple textbox of sorts. Anything else will require a custom widget.
At this point you have a very minimal browser. Adding CSS, javascript, etc requires that you include a pipeline to route data through the proper channels before it reaches the GUI. This is where the bulk of the work goes. Bring beer. :)
- Write the networking code to retrieve data from servers. I'd start with GET requests to keep it simple. This part is not that bad and its actually pretty fun. Doing it in Python is very simple and straightforward.
- Write browser GUI because its a native application. This depends on what language you use for the task. If Python, you can get away with tkinter (flame suit on) for a very simple prototype. I believe tkinter is single threaded and that may turn out to be an issue (don't remember).
- Write a simple rendering engine to display the text. No need to do this inside of the GUI code because the output can be read through the console. You are pretty much reading the raw html and extracting the bits you want.
- Plug in the rendering engine into a GUI widget. Since this is a (assumed) read-only browser, you can get away with a simple textbox of sorts. Anything else will require a custom widget.
At this point you have a very minimal browser. Adding CSS, javascript, etc requires that you include a pipeline to route data through the proper channels before it reaches the GUI. This is where the bulk of the work goes. Bring beer. :)
You have mainly two options: do everything from scratch or reuse. Starting from scratch is almost impossible to do alone, just Chrome was developed by different teams with different expertises, some of them acquired from other companies.
The second option is reusing a web component or browser and modify it for your purposes. Building on top of Chrome or Firefox (or their variants) is a good choice. I think Chrome is better engineered than Firefox and the code is clear. Obviously this is just an opinion.
Now the question is what do you want to add to such browser? Is this something you can add via an extension or is something that requires a new direction?
The second option is reusing a web component or browser and modify it for your purposes. Building on top of Chrome or Firefox (or their variants) is a good choice. I think Chrome is better engineered than Firefox and the code is clear. Obviously this is just an opinion.
Now the question is what do you want to add to such browser? Is this something you can add via an extension or is something that requires a new direction?
>Chrome was developed by different teams with different expertises, some of them acquired from other companies.
To expand on this, Chrome and Safari use webkit as their rendering/layout engine which is a fork of KDE's KHTML. IE came from NCSA Mosaic-> Spyglass -> Microsoft IE.
To be clear on my suggestions below, let's clarify what a browser actually is. The browser manages things like the navigation bar, bookmarks, download/mime handling, etc. This is known as the "chrome" or GUI. The layout engine handles HTML parsing, rendering, possibly is coupled with a Javascript engine like V8.
If you are trying to write a browser, the answer is to use webkit2 in whatever language you like. It's fast, light, and portable. If you are trying to write a layout engine (rendering engine), do as another poster suggested and write one to target HTML 3.2.
To expand on this, Chrome and Safari use webkit as their rendering/layout engine which is a fork of KDE's KHTML. IE came from NCSA Mosaic-> Spyglass -> Microsoft IE.
To be clear on my suggestions below, let's clarify what a browser actually is. The browser manages things like the navigation bar, bookmarks, download/mime handling, etc. This is known as the "chrome" or GUI. The layout engine handles HTML parsing, rendering, possibly is coupled with a Javascript engine like V8.
If you are trying to write a browser, the answer is to use webkit2 in whatever language you like. It's fast, light, and portable. If you are trying to write a layout engine (rendering engine), do as another poster suggested and write one to target HTML 3.2.
Note that Chrome is actually using Blink now, which itself is a fork of webkit
Servo https://github.com/servo/servo is a browser rewrite in Rust. Could be a good place to start.
Or a good study if the the goal language is different. Studying a clean codebase is where I'd start, there is likely a lot of tribal knowledge in HTML rendering (especially layout) that isn't documented anywhere but in code.
chrome/blink was built on webkit which was a fork of khtml. none of it was from scratch within the last decade.
This series of blog posts might be a good place to start:
https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h...
This series was really well-written and fun to work through! I second this recommendation.
As somebody already wrote "Starting from scratch is almost impossible to do alone". However you could set a first goal of doing some basic rendering of plain HTML, even text based. That should make you familiar with the hurdles of a rendering engine. Then apply some CSS. Even writing a browser for HTML 3.2 from scratch and no JavaScript should be an exercise to be proud of https://www.w3.org/TR/REC-html32 (1997). Then start looking at modern browsers using the links others provided. You should have the experience to understand all the moving parts by then.
This is similar to writing the very first web application without using any framework, maybe even interfacing directly with the web server. I remember decoding CGI arguments in C in 1994 before discovering Perl's CGI.pm. Then you understand the magic inside frameworks and don't get burned by surprises.
This is similar to writing the very first web application without using any framework, maybe even interfacing directly with the web server. I remember decoding CGI arguments in C in 1994 before discovering Perl's CGI.pm. Then you understand the magic inside frameworks and don't get burned by surprises.
I did this once back in 2003. It was text only rendered, with mouse support so you could click on the links instead of tabbing around like Links.
I built it off of the IE object model (Internet Explorer Browser Extensions? Can’t recall). I think I just used the object model for connectivity settings. If you could get to the net using IE, my browser would work as well.
I think I coded to HTML 1.1 specs, and had it working after 2 months (with a few minor rendering issues). It did not support scripting, CSS or really much of anything (although it could POST), but it was surprising how much of the web was accessible. And it was really fast. Not because of my skillz, but when you cut out the garbage, it’s amazing how fast a page will load.
Anyway, I would not recommend this approach, but I would offer you encouragement. HTML parsing isn’t really that big of a deal, and you could get something very minimal (text rendering) working in short order. It would at least give you a feeling of what you are up against.
I built it off of the IE object model (Internet Explorer Browser Extensions? Can’t recall). I think I just used the object model for connectivity settings. If you could get to the net using IE, my browser would work as well.
I think I coded to HTML 1.1 specs, and had it working after 2 months (with a few minor rendering issues). It did not support scripting, CSS or really much of anything (although it could POST), but it was surprising how much of the web was accessible. And it was really fast. Not because of my skillz, but when you cut out the garbage, it’s amazing how fast a page will load.
Anyway, I would not recommend this approach, but I would offer you encouragement. HTML parsing isn’t really that big of a deal, and you could get something very minimal (text rendering) working in short order. It would at least give you a feeling of what you are up against.
My 2 Cents:
Start out with a solid base: (for example)
- https://cefbuilds.com/
- https://crosswalk-project.org/
- http://electron.atom.io/
- http://nwjs.io/
or you try to help with existing projects: (for example)
- https://github.com/breach/thrust (started out with https://github.com/breach/breach_core a "javascript"-Browser (core is not js))
https://github.com/minbrowser/min
Start out with a solid base: (for example)
- https://cefbuilds.com/
- https://crosswalk-project.org/
- http://electron.atom.io/
- http://nwjs.io/
or you try to help with existing projects: (for example)
- https://github.com/breach/thrust (started out with https://github.com/breach/breach_core a "javascript"-Browser (core is not js))
https://github.com/minbrowser/min
Breach seems like a poor choice since the project has been inactive for ~2 years.
Maybe you're right, a project like servo (https://github.com/servo/servo) looks like a better fit.
easy bugs to hunt: https://github.com/servo/servo/labels/E-easy
easy bugs to hunt: https://github.com/servo/servo/labels/E-easy
[deleted]
Browser Engine https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h...
How to Create your Own Internet Browser in 5 Minutes! - https://www.youtube.com/watch?v=Bn22JBRC1UI
Chrome's Engine http://www.chromium.org/blink
How to Create your Own Internet Browser in 5 Minutes! - https://www.youtube.com/watch?v=Bn22JBRC1UI
Chrome's Engine http://www.chromium.org/blink
"If you wish to make a web browser from scratch, you must first invent the universe."
There is a "Building a Web Browser" course on udacity[1]. Maybe someone who has taken the course comment on it.
[1] https://www.udacity.com/course/programming-languages--cs262
[1] https://www.udacity.com/course/programming-languages--cs262
If you are seeking tutorials, or UI libraries etc, sorry, I can't help you with that as I am unaware of those things. However, if you are asking from which aspect to start building one from scratch, I would say start with layouting. Try implementing a tiny subset of div related things like position and display and see if this is what you want to spend your time working on. Even if you can crack this thing easily, remember, this is just the tip of the iceberg. I am not discouraging you, but know that you would have to digest the entire DOM and HTML spec, not to mention have a JavaScript engine and implement all the security related intricacies, at the least, for your software to become a web browser.
I'm curious: is there a ready-to-use set of tests for a layout engine that checks conformance with the specs? Something that you could, without much effort, run against a toy layout engine you'd write?
I know Webkit/Blink have humongous test suites, but these probably aren't easy to adapt to a toy layout engine.
I know Webkit/Blink have humongous test suites, but these probably aren't easy to adapt to a toy layout engine.
There's something like Acid3, but it doesn't give you information as to what failed when you get a less-than-perfect score. (Test: http://acid3.acidtests.org/ | Information: http://www.webstandards.org/action/acid3/)
There are also all those other browser benchmarks (e.g. http://peacekeeper.futuremark.com/), but those include things other than the layout engine, such as javascript performance, webGL, etc. You could try poking around a few of them and only looking at the CSS and DOM sections though (http://web.basemark.com/).
There are also all those other browser benchmarks (e.g. http://peacekeeper.futuremark.com/), but those include things other than the layout engine, such as javascript performance, webGL, etc. You could try poking around a few of them and only looking at the CSS and DOM sections though (http://web.basemark.com/).
W3C Web Platform Tests probably would be best starting point.
http://testthewebforward.org/
http://testthewebforward.org/
Back when I did CS 101 this was our final project; the teacher made the HTML-parser, and we students did the renderer and UI.
The HTML-parser is probably the part it makes sense to do first.
What language and platform do you plan to use? I think a modern OO-language will be helpful.
The HTML-parser is probably the part it makes sense to do first.
What language and platform do you plan to use? I think a modern OO-language will be helpful.
Just a note: The browser as we use it today is a result of 20+ years of human work and innovation. It's what you would call a killer application.
Equally - the car we drive today is a result of 130 years of human work and innovation. But after all that time we're seeing a real shake with the electric car. Things change when people try. I say good luck to the girl/guy - perhaps they'll create something incredible :-)
Even the electric car didn't start from scratch - it'll use the same component manufacturers as the rest of the car industry - e.g. tyres, brakes, suspension, interior, safety systems.
You and parent poster both make good points.
Well, this is what I could do in that (or almost any) case:
1) Setting features, key points, points to avoid, and goals of the project.
2) Choosing the toolchain to use, for resolving the points at 1)
3) Coding the features with that toolchain
Step 2, may involve: to read a lot (doc, rfc's, specifications, using search engines with keywords, search for "problems", etc) to look at other people's source code of similar programs, to try other people's tools and libraries and evaluate and compare them, to make integration tests, and to make Proofs Of Concept prototypes.
The toolchain (programing language, user GUI, etc) comes with an ecosystem (I hope you don't aim at rewriting also your $lang libraries for validate XML or interact with devices), so you have to choose looking at both.
To separate steps 2 and 3, use iterations, while you have resources.
If you find an opensource product that full-fills all your definitions of point 1), except a few of them, you can evaluate to start by a patch and a pull-request to them, or a fork :P
1) Setting features, key points, points to avoid, and goals of the project.
2) Choosing the toolchain to use, for resolving the points at 1)
3) Coding the features with that toolchain
Step 2, may involve: to read a lot (doc, rfc's, specifications, using search engines with keywords, search for "problems", etc) to look at other people's source code of similar programs, to try other people's tools and libraries and evaluate and compare them, to make integration tests, and to make Proofs Of Concept prototypes.
The toolchain (programing language, user GUI, etc) comes with an ecosystem (I hope you don't aim at rewriting also your $lang libraries for validate XML or interact with devices), so you have to choose looking at both.
To separate steps 2 and 3, use iterations, while you have resources.
If you find an opensource product that full-fills all your definitions of point 1), except a few of them, you can evaluate to start by a patch and a pull-request to them, or a fork :P
If you wanted to choose how to build the UI, one starting point is here https://news.ycombinator.com/item?id=12376242
Do you want to build a web browser or a web browser engine?
If you just want to build a web browser, here's my request for a new browser for iOS since web browsing on iOS is a second or third rate experience:
- Built-in ad-blocking.
- A way to disable JavaScript immediately for any website.
- A desktop mode that actually works, so that YouTube, Reddit, Imgur and other sites don't redirect me to their crappy mobile site without asking.
- Launches a completely separate app for private browsing.
Right now I use Dolphin browser on iOS and despite being in desktop mode, I still have to refresh every YouTube page that I land on in order to get the desktop style native HTML5 video player that allows me to go full-screen. On Imgur, gifv files don't play - I have to change the extension to gif and then wait 60 seconds as it downloads. Lots of other sites have similar types of issues. Meanwhile, gifv files work fine in Brendan Eich's "Brave" browser. It could be as easy to fix as just changing the user-agent.
Dolphin has good ad-blocking, but there is no way to quickly turn off JavaScript for a given page or domain (ala Quick JavaScript Switcher for Chrome).
The thing that really annoys me though about Dolphin is that Private Browsing does not work at all. That's probably because they want you to buy Dolphin Zero for $2.99.
If you just want to build a web browser, here's my request for a new browser for iOS since web browsing on iOS is a second or third rate experience:
- Built-in ad-blocking.
- A way to disable JavaScript immediately for any website.
- A desktop mode that actually works, so that YouTube, Reddit, Imgur and other sites don't redirect me to their crappy mobile site without asking.
- Launches a completely separate app for private browsing.
Right now I use Dolphin browser on iOS and despite being in desktop mode, I still have to refresh every YouTube page that I land on in order to get the desktop style native HTML5 video player that allows me to go full-screen. On Imgur, gifv files don't play - I have to change the extension to gif and then wait 60 seconds as it downloads. Lots of other sites have similar types of issues. Meanwhile, gifv files work fine in Brendan Eich's "Brave" browser. It could be as easy to fix as just changing the user-agent.
Dolphin has good ad-blocking, but there is no way to quickly turn off JavaScript for a given page or domain (ala Quick JavaScript Switcher for Chrome).
The thing that really annoys me though about Dolphin is that Private Browsing does not work at all. That's probably because they want you to buy Dolphin Zero for $2.99.
I checked out Brave yesterday for the first time and I think it does most of those things.
Brave has a desktop mode that mostly works and a built-in adblocker.
It does not have a `quick JS disable` button.
It also doesn't have a speed dial that I can manually add items to, which is one very important item that I forgot to mention earlier.
What I really want to do is make an open source browser that I can compile and put on my iOS devices myself too, without going to the app store. Then I could add all sorts of features and extensions without having to ask Apple.
It does not have a `quick JS disable` button.
It also doesn't have a speed dial that I can manually add items to, which is one very important item that I forgot to mention earlier.
What I really want to do is make an open source browser that I can compile and put on my iOS devices myself too, without going to the app store. Then I could add all sorts of features and extensions without having to ask Apple.
3 dollars? It really annoys you and you can't be bothered to spend 3 dollars?
For a web browser? You really want to pay 3 dollars for a web browser?
Also - Are you really surprised that people balk at paying for things that were once free? Really????
Furthermore, Dolphin has a "Private Mode" and it does not work at all. It's a lie. They are clearly trying to steer you towards their Zero browser. I don't give 3 dollars to companies who try to play me like that.
Nope. No, no way. Uh-uh. Forget it. For now, I just use Brave for private browsing and Dolphin for public browsing.
I actually want an open source one and I'm thinking about building it myself too. Looks like Brave is open source AND it's built with Swift which is exactly what I was going to do. I can just take that and rip out all the crap and add what I want.
Also - Are you really surprised that people balk at paying for things that were once free? Really????
Furthermore, Dolphin has a "Private Mode" and it does not work at all. It's a lie. They are clearly trying to steer you towards their Zero browser. I don't give 3 dollars to companies who try to play me like that.
Nope. No, no way. Uh-uh. Forget it. For now, I just use Brave for private browsing and Dolphin for public browsing.
I actually want an open source one and I'm thinking about building it myself too. Looks like Brave is open source AND it's built with Swift which is exactly what I was going to do. I can just take that and rip out all the crap and add what I want.
I'd say, if you have the opportunity and want to go wild skip the DOM and create an equivalent model using OpenGL. All the other Web platform features like the distribution model, resource streaming etc can still be used. It's just that the DOM is the great bottleneck and most of the hours devoted to improve the Web are being wasted on sorting quirks related to the DOM. Maybe yours will be the starting point towards a DOM-free Web! :)
Well, what are your goals?
You can accept entered URLs and fetch & display web pages quite easily nowadays. You should, depending on the vertical, be very concerned about security.
What are your goals (to a greater degree than "web browswer")?
What are your goals (to a greater degree than "web browswer")?
1. Write a webassembly interpreter/vm; (main step)
2. Using webassembly, write an html interpreter/renderer;
3. Using webassembly, write an css, js ... interpreter/renderer;
4. Keep doing this for all w3c standards for the rest of your life.
Chances are that by the time you are somewhere in the middle of step 2, people will have written the other interpreters in webassembly and the result of step 1 will be a functional browser already.
2. Using webassembly, write an html interpreter/renderer;
3. Using webassembly, write an css, js ... interpreter/renderer;
4. Keep doing this for all w3c standards for the rest of your life.
Chances are that by the time you are somewhere in the middle of step 2, people will have written the other interpreters in webassembly and the result of step 1 will be a functional browser already.
Step one: Build AI or machine learning or genetic code to build a browser by replicating the results of a real world browser.
Step two: run it!
Step two: run it!
[deleted]
I'm working on one right now. I'm using the electron shell because im not worried about the backend source (chromium) I'm just working on a new ui and a reader feature.
Email at [email protected] if you want to get a beta
I sent you an email.
Well, I did exactly that.
Sciter ( http://sciter.com ) is an embeddable HTML/CSS/script engine that has all components of a browser - html, css, svg, script and networking (http,ajax,websockets,etc).
Work started in 2002 when I decided to create WYSIWYG HTML editor for my wife. So http://blocknote.net/ was born. That was pure HTML (no CSS) rendering and editing engine. 9 months for one developer - myself.
After its publishing joined initial EverNote team to create WYSIWYG editor for EverNote. Three of us (developers), + Stepan Pachikov that generated the idea of the EverNote, created first published version of the EverNote (v 2.0). That was also done in 9 months (I am pretty sure that 9 months is not a coincidence but rather a law for new projects - optimal period when developers are excited of what they are doing).
While working on it, and being a UI-in-all-its-incarnations developer/architect, I've discovered that HTML and CSS are quite convenient format for defining UI layouts and elements. So next step was wrapping it as a reusable component and so HTMLayout engine was born ( http://htmlayout.com ).
Almost immediately discovered that plain C++ is not quite convenient for handling complex UI cases - "on click here hide that and expand those". UI objects ownership graph is quite complex and contains loops. Something with GC is definitely required. So decided to add scripting to it. Considered quite many existing languages including JS for the "language behind UI". At the end decided to roll my own tiscript ( http://www.codeproject.com/Articles/33662/TIScript-Language-... ) that essentially is JavaScript++. So 7 years ago I had what we will have in ES6 in next couple of years.
At the mean time I participated in HTML5 spec. development at W3C as an invited expert. And worked as an architect / team lead in several Web Application projects that required deep understanding of the platform - knowing Web UI stack from inside-out helped that a lot.
As Sciter works on all major desktop and mobile platforms I've created core engine wrappers for Windows API, MacOS CoreFoundation, Linux/XWindows/GTK and some others. Sciter core is written in C++ with Objective C fragments in Apple's wrappers. GFX technologies: GDI, GDI+, AGG, Direct2D, CoreGraphics, GTK, OpenGL, DirectX and others.
Script in Sciter is not quite a JS so I've created ports of jQuery (zepto feature set) and AngularJS alike data-binding mechanism based on Object.observe() that Sciter has.
More of Sciter's history: http://sciter.com/10-years-road-to-sciter/
At the moment Sciter code works on around 350 mln PCs and Mac's worldwide as part of various products.
Advice: all this is really possible if you have real motivation to do all this. Estimate that first. Split the task on 6-9 months periods. Start from creating basic DOM and CSS rendering. You can do that without parsers initially. Parser are boring - postpone them for the moment when you will have something breathing. Just a small trick for self-motivation.
Other than that it is doable, yes. For the simple reason: as smaller dev team as higher its productivity. Solo developer has highest productivity possible. Some conditions apply of course.
Sciter ( http://sciter.com ) is an embeddable HTML/CSS/script engine that has all components of a browser - html, css, svg, script and networking (http,ajax,websockets,etc).
Work started in 2002 when I decided to create WYSIWYG HTML editor for my wife. So http://blocknote.net/ was born. That was pure HTML (no CSS) rendering and editing engine. 9 months for one developer - myself.
After its publishing joined initial EverNote team to create WYSIWYG editor for EverNote. Three of us (developers), + Stepan Pachikov that generated the idea of the EverNote, created first published version of the EverNote (v 2.0). That was also done in 9 months (I am pretty sure that 9 months is not a coincidence but rather a law for new projects - optimal period when developers are excited of what they are doing).
While working on it, and being a UI-in-all-its-incarnations developer/architect, I've discovered that HTML and CSS are quite convenient format for defining UI layouts and elements. So next step was wrapping it as a reusable component and so HTMLayout engine was born ( http://htmlayout.com ).
Almost immediately discovered that plain C++ is not quite convenient for handling complex UI cases - "on click here hide that and expand those". UI objects ownership graph is quite complex and contains loops. Something with GC is definitely required. So decided to add scripting to it. Considered quite many existing languages including JS for the "language behind UI". At the end decided to roll my own tiscript ( http://www.codeproject.com/Articles/33662/TIScript-Language-... ) that essentially is JavaScript++. So 7 years ago I had what we will have in ES6 in next couple of years.
At the mean time I participated in HTML5 spec. development at W3C as an invited expert. And worked as an architect / team lead in several Web Application projects that required deep understanding of the platform - knowing Web UI stack from inside-out helped that a lot.
As Sciter works on all major desktop and mobile platforms I've created core engine wrappers for Windows API, MacOS CoreFoundation, Linux/XWindows/GTK and some others. Sciter core is written in C++ with Objective C fragments in Apple's wrappers. GFX technologies: GDI, GDI+, AGG, Direct2D, CoreGraphics, GTK, OpenGL, DirectX and others.
Script in Sciter is not quite a JS so I've created ports of jQuery (zepto feature set) and AngularJS alike data-binding mechanism based on Object.observe() that Sciter has.
More of Sciter's history: http://sciter.com/10-years-road-to-sciter/
At the moment Sciter code works on around 350 mln PCs and Mac's worldwide as part of various products.
Advice: all this is really possible if you have real motivation to do all this. Estimate that first. Split the task on 6-9 months periods. Start from creating basic DOM and CSS rendering. You can do that without parsers initially. Parser are boring - postpone them for the moment when you will have something breathing. Just a small trick for self-motivation.
Other than that it is doable, yes. For the simple reason: as smaller dev team as higher its productivity. Solo developer has highest productivity possible. Some conditions apply of course.
[deleted]
By giving up? ;-)
Instrument an existing browser and synthetize a new layout engine with machine learning and superoptimisation.
Sounds like a pretty good interview question. Better than asking you to invert binary trees or similar WTF. They're testing your judgment and asking you to assess your own skills and possibly they're interested in the actual assessment results assuming you're any good at assessment.
Most people would generally break the task chronologically or kinda data flow oriented. Probably. Now should you start at the part you're familiar with or less familiar or unfamiliar with? My philosophy is its a bathtub shaped curve and if you know the topic dead cold or have absolutely no idea how to do it, start there because those will be your constraints, either you are an industry expert on a topic and can define the constraints with the best of them in which case you should definitely start there unless you suspect a trick question (LOL you thought we meant HTML browser, actually we're passing you DB2 ISAM files and COBOL PIC statements), or you have absolutely no idea what you're doing in which case you should start there to begin to discover your constraints and APIs and interfaces and data structures. Never start at a point where you have intermediate level of knowledge, the impact when you smack into the unknown is going to be painful so start in the unknown to begin with.
Basically, it boils down to try not to paint yourself into a corner, so either start painting in the most obvious safe area to start or the most mysterious area on the assumption escape will be the easiest. But never, ever start to paint a floor by just dumping the can out in the middle of the floor.
If you have a flat level of skill across the entire project (yeah not likely) then if its a "simple" project like a viewer/browser where you've seen all the parts before, start in the middle and work both directions at once if you can handle the inevitable mystifying weirdness at both ends simultaneously, and if its a "complex" project like replace the entire air traffic control system then start at one edge and work to the other to minimize simultaneous confusion. Yeah yeah I know I just said never start painting in the middle of the floor, well if you think there's a perfectly flat skill level you probably haven't analyzed the problem enough so the real answer is not to start at all, and instead to think about it a lot more.
Of course half the fun is defining the starting point. Do they give you a working example to clone as a goal, access to all RFCs, no docs and cleanroom reimplement using a protocol analyzers traces, can you use a language you like, can you use language libraries that implement some fraction of the problem, does it have to be a 1995 level of browser or 2015 or 2035 level of browser ...
To some extent its a meta interview question where they want to see if you start talking like you're a system architect or systems analyst type like above or if you're a security guy you'd probably start with "review all the latest exploits and avoid all of them simultaneously (good luck lol)" or if you're a code monkey maybe you'd start with a rant about your favorite font to use in Eclipse and how nice your mechanical keyboard is or maybe you're not even a programmer in which case the response will sound like the dude from the office space movie in the scene arguing about his radio "well I was informed this was a programming test and I got permission to list the integers from 1 to 100 and print fizz when they're a multiple of ..." and so on.
Most people would generally break the task chronologically or kinda data flow oriented. Probably. Now should you start at the part you're familiar with or less familiar or unfamiliar with? My philosophy is its a bathtub shaped curve and if you know the topic dead cold or have absolutely no idea how to do it, start there because those will be your constraints, either you are an industry expert on a topic and can define the constraints with the best of them in which case you should definitely start there unless you suspect a trick question (LOL you thought we meant HTML browser, actually we're passing you DB2 ISAM files and COBOL PIC statements), or you have absolutely no idea what you're doing in which case you should start there to begin to discover your constraints and APIs and interfaces and data structures. Never start at a point where you have intermediate level of knowledge, the impact when you smack into the unknown is going to be painful so start in the unknown to begin with.
Basically, it boils down to try not to paint yourself into a corner, so either start painting in the most obvious safe area to start or the most mysterious area on the assumption escape will be the easiest. But never, ever start to paint a floor by just dumping the can out in the middle of the floor.
If you have a flat level of skill across the entire project (yeah not likely) then if its a "simple" project like a viewer/browser where you've seen all the parts before, start in the middle and work both directions at once if you can handle the inevitable mystifying weirdness at both ends simultaneously, and if its a "complex" project like replace the entire air traffic control system then start at one edge and work to the other to minimize simultaneous confusion. Yeah yeah I know I just said never start painting in the middle of the floor, well if you think there's a perfectly flat skill level you probably haven't analyzed the problem enough so the real answer is not to start at all, and instead to think about it a lot more.
Of course half the fun is defining the starting point. Do they give you a working example to clone as a goal, access to all RFCs, no docs and cleanroom reimplement using a protocol analyzers traces, can you use a language you like, can you use language libraries that implement some fraction of the problem, does it have to be a 1995 level of browser or 2015 or 2035 level of browser ...
To some extent its a meta interview question where they want to see if you start talking like you're a system architect or systems analyst type like above or if you're a security guy you'd probably start with "review all the latest exploits and avoid all of them simultaneously (good luck lol)" or if you're a code monkey maybe you'd start with a rant about your favorite font to use in Eclipse and how nice your mechanical keyboard is or maybe you're not even a programmer in which case the response will sound like the dude from the office space movie in the scene arguing about his radio "well I was informed this was a programming test and I got permission to list the integers from 1 to 100 and print fizz when they're a multiple of ..." and so on.
[Disclaimer: I'm not an expert on developing browsers] It depends on what exactly you'd want to implement, since a (modern) browser is really a huge piece of software with several aspects and components in it. Looking at it from one angle, you could split it into chrome (not Google Chrome, but the UI, like tab bar, bookmarks, menus, extensions management, etc.) and content (the rendering engine).
You'd most likely get bogged down if you start writing your own rendering engine (processing HTML, CSS and JS, and processing image formats, video and sound formats, even not including any plugins), unless it's a very limited scope toy browser that can handle some simple stuff. In this case, start with base HTML support, including handling incomplete tags and quirks like that have been carried over for decades (the amount of cruft in rendering engines for backward compatibility and handling undocumented but accepted quirks would itself take a long time to develop). Then try a little bit of CSS support. I'd assume JS support could be quite challenging since that moves out of the already challenging "paint the screen" part into an even more challenging part on interpreting code, running it, optimizing it, etc., which is a lot more complex when you consider there would be several JS files loaded on most websites.
Something a little more easier (only comparatively) would be developing the chrome around an existing rendering engine and seeing how to handle tabs, background loading, foreground loading, bookmarks, private/incognito browsing, etc., and if you feel adventurous enough, adopt a particular process or threading model for a multi-tab browser.
Another alternative here is to create a WebView application and see which parts you could improve. For example, Chrome on iOS and Firefox on iOS are just the chrome for the built-in rendering engine in iOS, but address many things like tabs, networking, incognito mode, bookmarks, sync and other features. You could try creating a similar one on iOS, latching on to the work done by FoxBrowser [1] or Firefox for iOS [2]. On Android, you could try building on top of Firefox for Android [3].
If you'd like to work on a web browser rendering engine that's a bit more closer to "from scratch", I'd recommend working with the Servo [4] team and seeing how you can help. It's a long drawn project that has already made huge strides, but you'd get an inside view of the complexity of developing a browser engine. What's more, you could also learn Rust while doing this. [5]
[1]: https://github.com/graetzer/Foxbrowser
[2]: https://github.com/mozilla/firefox-ios
[3]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_g...
[4]: https://servo.org/
[5]: https://www.rust-lang.org/
You'd most likely get bogged down if you start writing your own rendering engine (processing HTML, CSS and JS, and processing image formats, video and sound formats, even not including any plugins), unless it's a very limited scope toy browser that can handle some simple stuff. In this case, start with base HTML support, including handling incomplete tags and quirks like that have been carried over for decades (the amount of cruft in rendering engines for backward compatibility and handling undocumented but accepted quirks would itself take a long time to develop). Then try a little bit of CSS support. I'd assume JS support could be quite challenging since that moves out of the already challenging "paint the screen" part into an even more challenging part on interpreting code, running it, optimizing it, etc., which is a lot more complex when you consider there would be several JS files loaded on most websites.
Something a little more easier (only comparatively) would be developing the chrome around an existing rendering engine and seeing how to handle tabs, background loading, foreground loading, bookmarks, private/incognito browsing, etc., and if you feel adventurous enough, adopt a particular process or threading model for a multi-tab browser.
Another alternative here is to create a WebView application and see which parts you could improve. For example, Chrome on iOS and Firefox on iOS are just the chrome for the built-in rendering engine in iOS, but address many things like tabs, networking, incognito mode, bookmarks, sync and other features. You could try creating a similar one on iOS, latching on to the work done by FoxBrowser [1] or Firefox for iOS [2]. On Android, you could try building on top of Firefox for Android [3].
If you'd like to work on a web browser rendering engine that's a bit more closer to "from scratch", I'd recommend working with the Servo [4] team and seeing how you can help. It's a long drawn project that has already made huge strides, but you'd get an inside view of the complexity of developing a browser engine. What's more, you could also learn Rust while doing this. [5]
[1]: https://github.com/graetzer/Foxbrowser
[2]: https://github.com/mozilla/firefox-ios
[3]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_g...
[4]: https://servo.org/
[5]: https://www.rust-lang.org/
TempleOS
1) Write socket code to retrieve the page from a server
2) Get it to draw the text in a window.
3) Write code to look for tags and start diving up the retrieved file into a proper DOM tree. Perhaps start by creating an object for each paragraph. Then maybe look for <b> tags, then <div> tags
4) Change the rendering code to draw the screen for the data structure you created above.
5) Iterate a lot. Add support for parsing more and more tags into the data model. Start adding annotations for various attributes.
6) Start improving the layout code. Draw things in <b> tags in bold. Draw things in <div> tags under each other and <span> tags separately.
7) Add limited CSS support. Allow just width and height and border sections. make the code read the css file and attach the attributes to the correct sections of the DOM document you are creating.
8) Improve the layout code to look at the width and height tags on each element, and where tags are nested to propogate the width and height information up and down the tree as needed. Draw borders if the css tells you to. Look at font weight etc.
9) Iterate. Add one feature at a time. Repeat for a year until you have an browser that can render basic pages.
10) Iterate some more. Read specs. Rewrite your document to DOM tree parse a number of times perhaps using a more formal grammar. You'll probably be on about year 8 by now.
11) Add javascript support... :P
Perhaps my point is though that this seems something where its very easy to start small, just render the text of a page, and incrementally add features and improve the layout. It will take a long time though as there are so many features to add.