New open source project: Common Lisp 3D graphics system(old.reddit.com)
old.reddit.com
New open source project: Common Lisp 3D graphics system
https://old.reddit.com/r/lisp/comments/weocc5/new_open_source_common_lisp_3d_graphics_project/
82 comments
Ha ha, thanks.
I know 3D graphics, but I realize there are folks out there who know a lot more than I do in their respective fields. And I hope to tap into that knowledge.
It's going to take a team to do this project.
I know 3D graphics, but I realize there are folks out there who know a lot more than I do in their respective fields. And I hope to tap into that knowledge.
It's going to take a team to do this project.
You missed out "developed some of the earliest 3D CG software for Symbolics Lisp Machines"! That's like 10,000 nerd points right there :)
Not sure if it's nerd points or just old age. ;)
I went to the computer history museum on the weekend and took a pic of a symbolics machine for my workplaces slack that got a lot of love, I think there’s still some nerd points there :P
Ha ha. I talk about my Symbolics/MIT days in early episodes of my blog:
https://kaveh808.medium.com
Wish I still had the code, for sentimental reasons, but it all went up in a house fire.
https://kaveh808.medium.com
Wish I still had the code, for sentimental reasons, but it all went up in a house fire.
It's not entirely clear (and I think you even said that) where this is going. Tangentially, there is IMHO a great need for new FLOSS geometric kernel for CAD applications. The only option right now are OCCT: https://www.opencascade.com/open-cascade-technology/
which is really huge
Or the geometric code in Solvespace: https://solvespace.com/index.pl which is currently under 10,000 LoC: https://github.com/solvespace/solvespace/tree/master/src/srf This is far from perfect.
CAD systems use trimmed NURBS which have a lot of challenges. I wonder how choice of language would help to address some of the complexity. Addressing this is definitely a project unto itself, while having much narrower scope than a new Blender.
Or the geometric code in Solvespace: https://solvespace.com/index.pl which is currently under 10,000 LoC: https://github.com/solvespace/solvespace/tree/master/src/srf This is far from perfect.
CAD systems use trimmed NURBS which have a lot of challenges. I wonder how choice of language would help to address some of the complexity. Addressing this is definitely a project unto itself, while having much narrower scope than a new Blender.
I remember trimmed NURBS from back in the 80's, when the holy grail for 3D systems vendors was landing car companies as clients. It was not easy.
Later, at Square on the "Final Fantasy" movie, we were having trouble modeling our character models using NURBS due to continuity issues when more than four patches met at a corner. I resolved the issue by implementing a subdivision surface plugin for Maya. Overnight the workflow switched to polygons.
Would be great to have NURBS and other parametric representations as plugins in the system.
One of the most exciting things about this project will be seeing what gets developed on top of the system.
Later, at Square on the "Final Fantasy" movie, we were having trouble modeling our character models using NURBS due to continuity issues when more than four patches met at a corner. I resolved the issue by implementing a subdivision surface plugin for Maya. Overnight the workflow switched to polygons.
Would be great to have NURBS and other parametric representations as plugins in the system.
One of the most exciting things about this project will be seeing what gets developed on top of the system.
As far as this thread relates to lisp, I have written OpenCascade bindings for CL:
https://github.com/awolven/oc
A truly new geometric kernel would be great, and it's been a dream of mine, but I have too many other things that need to be finished before I start writing one.
https://github.com/awolven/oc
A truly new geometric kernel would be great, and it's been a dream of mine, but I have too many other things that need to be finished before I start writing one.
What would a new geometric kernel consist of? I'm not a CAD expert.
Usually when you talk about a geometric kernel in the past three decades, you're talking mainly about, but not exclusively about, a solid modeler.
A solid modeler generally consists of a trimmed curve and surface library with lots of topological reasoning, to be able to represent closed regions of space and manipulate them by adding holes and rounding features and offsets. The idea is to be able to represent and operate on just about any 3d shape imaginable. Solid modelers of recent years have had support for nurbs surfaces and polygonal subdivision surfaces, but also include support for conic sections, beziers and other implicit or parametric curves or surfaces. Many of them, like OpenCascade are arranged in a logical object oriented fashion which abstracts the complexity. They have to be general, featureful and fast for anybody to want to use them. It's a whole heck or a lot of math and geometric reasoning (and debugging) which makes them valuable (and huge), and although you might want to talk about a "new" kernel you would never just want to chuck something like OpenCascade and start over. OpenCascade, ACIS, Parasolid, SMLib and others encapsulate a huge corpus of knowledge no matter what language they are written in. I have taken criticism from other Lisp hackers for writing Lisp bindings to OpenCascade instead of "starting over" in Lisp, but these people really have no idea what they are talking about.
A solid modeler generally consists of a trimmed curve and surface library with lots of topological reasoning, to be able to represent closed regions of space and manipulate them by adding holes and rounding features and offsets. The idea is to be able to represent and operate on just about any 3d shape imaginable. Solid modelers of recent years have had support for nurbs surfaces and polygonal subdivision surfaces, but also include support for conic sections, beziers and other implicit or parametric curves or surfaces. Many of them, like OpenCascade are arranged in a logical object oriented fashion which abstracts the complexity. They have to be general, featureful and fast for anybody to want to use them. It's a whole heck or a lot of math and geometric reasoning (and debugging) which makes them valuable (and huge), and although you might want to talk about a "new" kernel you would never just want to chuck something like OpenCascade and start over. OpenCascade, ACIS, Parasolid, SMLib and others encapsulate a huge corpus of knowledge no matter what language they are written in. I have taken criticism from other Lisp hackers for writing Lisp bindings to OpenCascade instead of "starting over" in Lisp, but these people really have no idea what they are talking about.
Interesting. Sounds like a non-trivial undertaking. Both writing an interface or starting a new design. Either way, I would be happy if our system develops along to the point that serious engineering packages would want to use it.
The colors in the demos reminds me of Wings3d, a 3d editor written in Erlang. It had quite a unique user interface to it, with modal bindings not unlike Vim. Are the color choices a coincidence?
I based the menu design on the Symbolics menus, and Wings3D is a descendant of S-Geometry in spirit.
I wanted a quick way of assigning some colors so I just made a rainbow color ramp. If you mean the red color for selection, then yes that was inspired by watching a Wings3D video.
I wanted a quick way of assigning some colors so I just made a rainbow color ramp. If you mean the red color for selection, then yes that was inspired by watching a Wings3D video.
see also Nendo from Izware: https://archive.org/details/nendo-1-5-setup
which originally was a project for Nintendo..., IIRC
which originally was a project for Nintendo..., IIRC
If memory serves, Nendo was a standalone version of the S-Geometry modeler written in CL, and Wings3D was a clone of Nendo written in Erlang.
Mirai was in Lisp (ahh the repl on every error), but I'm not sure that Nendo was in Lisp.
Mirai was a descendant of your work? Nice. It was a great (at the time) software. Too bad Izware has dissapeared!
Mirai was a descendant of your work? Nice. It was a great (at the time) software. Too bad Izware has dissapeared!
You should start by building and shipping something even if it is minimal. Starting an open source project with a post calling to form a team and so on is an anti-pattern imo. Good OSS projects always should start with code from one or maybe two people that does something useful, and attract contributors from there. Just my 2c, good luck!
Thank you. I am thinking of refactoring my existing proof of concept to make it more portable. Currently it's hardwired for CCL and MacOS.
Then I may "ship" a release (maybe purely REPL-based) and gather feedback.
Then I may "ship" a release (maybe purely REPL-based) and gather feedback.
I agree with the GP that it's very difficult to get contributors if the project is not usable. Reading the post in reddit, it looks like just a call for help for an empty project.
But looking at the repo linked in the post https://github.com/kaveh808/kons-9 and the demo, it looks like you have something that is already working.
Also, what does this mean in the readme?
> If you're brave enough, change the pathnames in main.lisp and eval the buffer.
I see no pathnames in https://github.com/kaveh808/kons-9/blob/main/main.lisp I don't use CL, so perhaps I'm missing something, or it's an outdated instruction.
Another important thing to get users (and later contributor) is a complete idiot proof method to install the project and run the demo [1]. something like "git clone & make demo" or whatever is the correct syntax for that spell.
[1] I recommend to read an old post by Joel Spolsky https://www.joelonsoftware.com/2000/04/26/designing-for-peop...
But looking at the repo linked in the post https://github.com/kaveh808/kons-9 and the demo, it looks like you have something that is already working.
Also, what does this mean in the readme?
> If you're brave enough, change the pathnames in main.lisp and eval the buffer.
I see no pathnames in https://github.com/kaveh808/kons-9/blob/main/main.lisp I don't use CL, so perhaps I'm missing something, or it's an outdated instruction.
Another important thing to get users (and later contributor) is a complete idiot proof method to install the project and run the demo [1]. something like "git clone & make demo" or whatever is the correct syntax for that spell.
[1] I recommend to read an old post by Joel Spolsky https://www.joelonsoftware.com/2000/04/26/designing-for-peop...
I had hoped including a link to the repo would show that it wasn't an empty project, but I could probably have done better.
That instruction is indeed outdated. It is from before the system had ASDF.
I do need to update the Readme file.
I used to follow Joel's blog back in the day.
That instruction is indeed outdated. It is from before the system had ASDF.
I do need to update the Readme file.
I used to follow Joel's blog back in the day.
Yes I think you should spend some time cleaning up your PoC into a real, usable project. Write a nice readme, get it so anyone can run it quickly and easily, etc. In my most recent github project I released [1] I also used DALL-E to make a nice header to the README, which can also help make it feel like a solid thing to be worth checking out imo.
[1] https://github.com/gfodor/p2pcf
[1] https://github.com/gfodor/p2pcf
Cleaning up the code organization and making a release is number one on my list of development tasks.
It will also serve as a starting point for design discussions.
It will also serve as a starting point for design discussions.
Even if it is hard wired as long as it works and a lot of us using macOS, it is a start. Just a warning the whole thing will change is good.
Open is for others to be involved. In fact I am ok to play with Ccl and macOS.
Open is for others to be involved. In fact I am ok to play with Ccl and macOS.
If memory is reachable the surviving pictures of old Lisp Machines had inside color line diagrams where the memory is. Should Common Lisp and Khronos Vulkan API control color threads in the kernel space and that memory persists throughout the lifetime of the hardware will that open the door to a different style of "learning machine"?
Back in the 90s, Joel Welling and Chris Nuuja at the Pittsburgh Supercomputing Center at CMU developed a similar system called P3D.
It was really cool, and they had a bunch of backends to render on different high end graphic workstations. We even had a Pixar Renderman hooked up to a laser disk recorder. You could script a 3D scene with different light sources and cameras and over the course of a week, render some high quality NTSC video.
P3D: a Lisp-based format for representing general 3D models https://dl.acm.org/doi/10.5555/110382.110608
P3D DESCRIPTION AND DEFINITIONS http://netghost.narod.ru/gff/vendspec/p3d/p3d_desc.txt
P3D: a Lisp-based format for representing general 3D models https://dl.acm.org/doi/10.5555/110382.110608
P3D DESCRIPTION AND DEFINITIONS http://netghost.narod.ru/gff/vendspec/p3d/p3d_desc.txt
> over the course of a week, render some high quality NTSC video.
I remember those days. :)
I remember those days. :)
Was any of the actual code or other artifacts of the work preserved?
I looked, but I couldn't find a .tar.gz or anything.
I emailed Joel and asked him to put it up on his github page, https://github.com/jswelling
He has something called DrawP3D, but I think its just a library that you can call from C or Fortran that uses the P3D rendering backend. I could be wrong it was a long time ago.
I don't think it's entirely true that there's no documentation beyond the readme - aren't the code/concepts from your blog a reasonable introduction to people that want to jump in?
https://kaveh808.medium.com/
Previously on hn: https://news.ycombinator.com/item?id=31115221
https://kaveh808.medium.com/
Previously on hn: https://news.ycombinator.com/item?id=31115221
Good point. I never thought of the blog posts as documentation.
For a project about 3D graphics, I expect 3D graphics in the video! Lot of 3D graphics!
Can you make a tl;dw video? 1 minute, perhaps 3 minutes.
Some links to some arbitrarily selected parts of the video:
Something about (random?) particle flow: https://www.youtube.com/watch?v=NJe4isZ7NHI&t=1649s
A 3D graph a bull (and later the teapot): https://www.youtube.com/watch?v=NJe4isZ7NHI&t=2155s
Can you make a tl;dw video? 1 minute, perhaps 3 minutes.
Some links to some arbitrarily selected parts of the video:
Something about (random?) particle flow: https://www.youtube.com/watch?v=NJe4isZ7NHI&t=1649s
A 3D graph a bull (and later the teapot): https://www.youtube.com/watch?v=NJe4isZ7NHI&t=2155s
Good point. I need better marketing. Or a marketing team. :)
I'll work on a promotional video.
I'll work on a promotional video.
Calling all those interested in being a part of this.
When it comes to computing motions, and interpolating between motions, this might be a nice way of doing it: https://www.youtube.com/watch?v=pq9YfdPHhIo
I think that some computational geometry problems admit nice solutions using Projective Geometric Algebra. Using it is fairly intuitive: You have some elements of the algebra that represent Euclidean motions, and some other elements that represent flat objects like points, lines and planes; you can then intersect these objects, "join" them into larger objects, and apply transformations to them. If it looks to you like it could make some things you're doing easier, then you might try using it. I think there are already some Common Lisp implementations out there -- but if there aren't, then it shouldn't be difficult to make one from scratch.
There's also Conformal Geometric Algebra, which is notable for allowing you to represent circles and spheres as elements of the algebra. But efficiently implementing it appears to be harder than PGA.
I think that some computational geometry problems admit nice solutions using Projective Geometric Algebra. Using it is fairly intuitive: You have some elements of the algebra that represent Euclidean motions, and some other elements that represent flat objects like points, lines and planes; you can then intersect these objects, "join" them into larger objects, and apply transformations to them. If it looks to you like it could make some things you're doing easier, then you might try using it. I think there are already some Common Lisp implementations out there -- but if there aren't, then it shouldn't be difficult to make one from scratch.
There's also Conformal Geometric Algebra, which is notable for allowing you to represent circles and spheres as elements of the algebra. But efficiently implementing it appears to be harder than PGA.
I am quite interested in exploring new ways of doing 3D interpolated animation. Especially when it comes to visualizing and controlling the motion by the animator.
I have not yet seen a system or UI that I am happy with.
Having a flexible enough GUI toolset to allow this sort of development is definitely on my mind.
I have not yet seen a system or UI that I am happy with.
Having a flexible enough GUI toolset to allow this sort of development is definitely on my mind.
I think with such projects it is more important for the leaders to focus on the the higher level architecture, the data structure and algorithms and their proper documentation rather than the coding finished product.
Of course they can code when these details are fleshed out, but these traits in a project are what make it easier for more people to join and contribute.
My view is that there are more than enough people to make open source projects succeed, but trying to unravel and understand other peoples poorly documented architectural decisions and code is what make things difficult.
Of course they can code when these details are fleshed out, but these traits in a project are what make it easier for more people to join and contribute.
My view is that there are more than enough people to make open source projects succeed, but trying to unravel and understand other peoples poorly documented architectural decisions and code is what make things difficult.
Unrelated to the post. I like old.reddit.com which the story links to :)
Looks cool, although I don't understand much of graphics, I always felt, with lisp macros you can probably do bunch of cool things around graphics.
Looks cool, although I don't understand much of graphics, I always felt, with lisp macros you can probably do bunch of cool things around graphics.
I love Common Lisp but I feel that for a project like this to actually go somewhere it needs to have goals beyond the "we'd like to rewrite the software A in language L" trope.
There are many inspirations that we can take from CL development experience that might make for interesting directions to take 3D software systems. For example a live sharable image based environment would be sweet to allow artists to easily collaborate in real time.
I'll be keep an eye on this, I might have some time for CL projects in the future and would love to help out if possible.
There are many inspirations that we can take from CL development experience that might make for interesting directions to take 3D software systems. For example a live sharable image based environment would be sweet to allow artists to easily collaborate in real time.
I'll be keep an eye on this, I might have some time for CL projects in the future and would love to help out if possible.
You are absolutely right. And I don't have a logical argument as to why this will actually go somewhere. At best, all I can say is "If you build it, they might come."
I wrote about this in my blog a while back, if you can stomach the overthinking. :)
https://kaveh808.medium.com/15-interlude-musings-and-over-th...
Perhaps another way of putting it: This is what I want to spend my time doing.
I wrote about this in my blog a while back, if you can stomach the overthinking. :)
https://kaveh808.medium.com/15-interlude-musings-and-over-th...
Perhaps another way of putting it: This is what I want to spend my time doing.
You might be asked why a lot. And sometimes, I think, the best answer is 'because I can and I want to', as your article perfectly answers it. Being too utilitarian early in a project is sometimes useful but large, audacious bets don't work that way. Not all those who wander are lost, so do it!
Which perfectly fine from an individualistic perspective, and that's cool, but this was framed as a open project with call for community involvement.
A successful community project would benefit from a clear set of goals, which is oft times a matter of "software A in language L".
A successful community project would benefit from a clear set of goals, which is oft times a matter of "software A in language L".
Well put. Thank you.
Bless your heart for still using Common Lisp.
Been using it for almost all my personal projects for a few decades now. It's the only language that I don't feel is fighting me when I develop my code.
I used it exclusively for 4 years and had the same experience. I moved on so that I could work with people who are scared of parentheses on projects larger than what I could do by myself.
Working on a team with other Lisp-speaking aliens would be a dream.
Working on a team with other Lisp-speaking aliens would be a dream.
Yes, I know the feeling. My first job after my Symbolics/MIT experience was writing a 3D rendering system in Fortran. :)
That would be a dream. One I'm hoping to make happen here.
That would be a dream. One I'm hoping to make happen here.
[deleted]
Neat! Immediately made me think of Naughty Dog where they used Allegro CL to make Game Oriented Assembly Lisp (GOAL).
https://en.m.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
https://en.m.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
I find it ironic that instead of adopting ND's unique technology, Sony shut it down and went with standard fare. In business, the platform with the most available "developers" usually wins out.
It's very difficult to justify the training effort and cost for non-mainstream technology, even if it is demonstrably more productive. Easier to just hire twice as many staff.
It's very difficult to justify the training effort and cost for non-mainstream technology, even if it is demonstrably more productive. Easier to just hire twice as many staff.
Im my view the true reason is that knowledge based companies see their knowledge workers as their greatest threat and competition, so prefer to channel their employees energies into technologies which make it hard for their employees to use to compete against them.
If your employees are invested in Lisp rather than C++ how easy will it be for them to start up their own companies and compete with you if they need to hire an additional 10 C++ programmers rather than the 3 Lisp programmers they would require if you developed your product and thus their skillset in Lisp rather than C++?
After seeing very effective tools mothballed over the years this is the only reason I can come up with.
Just see how Javascript and HTML have been seen as the way of delivering program functionality to users over the last 25 years, and yet every two weeks a new Javascript framework is announced which promises to make web development easier.
I mean how many years does it take to develop effective frameworks for software development? 25 years is time enough for a people to develop from blobs in the parents bodies to university graduates and with families of their own, and yet there is no end to new frameworks which promise to solve all web software development issues.
If your employees are invested in Lisp rather than C++ how easy will it be for them to start up their own companies and compete with you if they need to hire an additional 10 C++ programmers rather than the 3 Lisp programmers they would require if you developed your product and thus their skillset in Lisp rather than C++?
After seeing very effective tools mothballed over the years this is the only reason I can come up with.
Just see how Javascript and HTML have been seen as the way of delivering program functionality to users over the last 25 years, and yet every two weeks a new Javascript framework is announced which promises to make web development easier.
I mean how many years does it take to develop effective frameworks for software development? 25 years is time enough for a people to develop from blobs in the parents bodies to university graduates and with families of their own, and yet there is no end to new frameworks which promise to solve all web software development issues.
My feeling is this is a consequence of the "worse is better" approach. Trying to shoehorn features which were not planned for in the initial design.
Which is why I always make the point most technology failures aren't because the tech wasn't up to it, rather human factor taking decisions.
Another good example was while Midori was powering the Asian Bing infrastructure, Windows team was still asserting it wasn't production ready (see Joe Duffy keynot on RustConf).
Our industry needs more dreamers and less bean counters.
Another good example was while Midori was powering the Asian Bing infrastructure, Windows team was still asserting it wasn't production ready (see Joe Duffy keynot on RustConf).
Our industry needs more dreamers and less bean counters.
One reason I'm going the open source route is to work with like-minded people and have the freedom to dream.
As I've probably made clear elsewhere in this posting, I can't make a business case for the project. Not one that an investor would buy.
As I've probably made clear elsewhere in this posting, I can't make a business case for the project. Not one that an investor would buy.
Please join the project GitHub repo if you would like to contribute or be notified of updates: https://github.com/kaveh808/kons-9
Your name reminds of one Kaveh Alamouti, PhD.
In a particular year in the late 80s or 90s he was the UK's highest earner by dint of bringing his mathematical genius to bear on the trading activities of Tokai Bank.
You might tap your namesake for some funding.
In a particular year in the late 80s or 90s he was the UK's highest earner by dint of bringing his mathematical genius to bear on the trading activities of Tokai Bank.
You might tap your namesake for some funding.
I'm afraid I don't know the gentleman. Or how an open source project could get funding.
for the lisp lover, https://github.com/inconvergent/weird is also great
Very cool. I'm always fascinated by DSLs for doing algorithmic art.
i have a personal opinion that any sufficiently large project written in a untyped language will become unreadable to anyone external to that project
Common Lisp is not untyped. It is dynamically typed.
Common Lisp is also object-oriented with classes, generic functions and methods.
Thus the source code will contain all the usual class and method declarations.
Common Lisp may also contain optional type declarations. For above example the slot BAR of the class FOO has a type NUMBER.
Common Lisp is also object-oriented with classes, generic functions and methods.
CL-USER 6 > (defclass foo ()
((bar :type number :initform 0)))
#<STANDARD-CLASS FOO 82200CF0DB>
CL-USER 7 > (defmethod add-number ((object foo) (n number))
(incf (slot-value object 'bar) n))
#<STANDARD-METHOD ADD-NUMBER NIL (FOO NUMBER) 80100E116B>
CL-USER 8 > (let ((foo1 (make-instance 'foo)))
(add-number foo1 10)
foo1)
#<FOO 80100E185B>
CL-USER 9 > (describe *)
#<FOO 80100E185B> is a FOO
BAR 10
Thus the source code will contain all the usual class and method declarations.
Common Lisp may also contain optional type declarations. For above example the slot BAR of the class FOO has a type NUMBER.
alright, taking my stone back to my yard! I didn't know CL was typed
You can type Common Lisp.
You can do anything* in Common Lisp. But will you?
* - almost anything
* - almost anything
Adding types in CL is pretty trivial [1] to do. So, yes, I will, and I do.
[1] https://lispcookbook.github.io/cl-cookbook/type.html#compile...
[1] https://lispcookbook.github.io/cl-cookbook/type.html#compile...
acalzycalzy(3)
> • wrote 3D animation software used in “Jurassic Park” • software R&D lead on “Final Fantasy: The Spirits Within” movie • senior software developer on “The Hobbit” films
> But I don't consider myself an expert
Looks like a cool project though!