Factor – A practical stack language(factorcode.org)
factorcode.org
Factor – A practical stack language
http://factorcode.org/
41 comments
I think you hit the nail on the head. I haven't actually used Factor, but I spent 4 years writing and debugging in a concatenative proprietary language, and this was my experience too. It was very productive to write in but required great discipline in terms of structure and documentation to be debuggable, and debugging felt very "interactive", where it was easier to catch problems live by watching the data than to solve them analytically. This put a lot of weight on the quality of the debugging tools, which were lacking to the point where it was all about logging values to a file with a tail log on it, and made it sometimes difficult to separate bad data from bad code, which put more weight on the specs being there. But when all the pieces were in place it was the most terse and beautiful code to read.
Factor used to have the stack displayed in a separate frame above the listener and it was removed at some point.
When I was using Factor I tended to keep the stack short and use the combinators to avoid having to think about large stacks. It worked pretty well.
When I was using Factor I tended to keep the stack short and use the combinators to avoid having to think about large stacks. It worked pretty well.
At risk of shameless self-promotion let me mention an interpreter for Joy I've written (in Python) https://code.google.com/p/joypy/
Joy is one of the major inspirations for Factor (I think it is considered the first "concatinative" language but I may be mistaken about that) and I think it deserves serious consideration from anyone interested in language design and formal systems. In a word, it's name is apt.
I have added a trace ability that lets the interpreter print out a complete description of the steps in an evaluation, like so (the bullet mark indicates the current interpreter "position", items to the left are on the stack, items to the right are the expression to be evaluated):
Joy is one of the major inspirations for Factor (I think it is considered the first "concatinative" language but I may be mistaken about that) and I think it deserves serious consideration from anyone interested in language design and formal systems. In a word, it's name is apt.
I have added a trace ability that lets the interpreter print out a complete description of the steps in an evaluation, like so (the bullet mark indicates the current interpreter "position", items to the left are on the stack, items to the right are the expression to be evaluated):
joy? 3 range_to_zero
# frame start
• 3 range_to_zero
3 • range_to_zero
# .. range_to_zero == unit [down_to_zero] infra
# .. frame start
3 • unit [down_to_zero] infra
# .... unit == [] cons
# .... frame start
3 • [] cons
3 [] • cons
[3] •
# .... frame end
# .... unit done.
[3] • [down_to_zero] infra
[3] [down_to_zero] • infra
# .... frame start
3 • down_to_zero
# ...... down_to_zero == [0 gt] [dup pred] while
# ...... frame start
3 • [0 gt] [dup pred] while
3 [0 gt] • [dup pred] while
3 [0 gt] [dup pred] • while
# ........ frame start
3 • 0 gt
3 0 • gt
True •
# ........ frame end
# ........ frame start
3 • dup pred
3 3 • pred
3 2 •
# ........ frame end
# ........ frame start
3 2 • 0 gt
3 2 0 • gt
3 True •
# ........ frame end
# ........ frame start
3 2 • dup pred
3 2 2 • pred
3 2 1 •
# ........ frame end
# ........ frame start
3 2 1 • 0 gt
3 2 1 0 • gt
3 2 True •
# ........ frame end
# ........ frame start
3 2 1 • dup pred
3 2 1 1 • pred
3 2 1 0 •
# ........ frame end
# ........ frame start
3 2 1 0 • 0 gt
3 2 1 0 0 • gt
3 2 1 False •
# ........ frame end
# ........ while done.
3 2 1 0 •
# ...... frame end
# ...... down_to_zero done.
3 2 1 0 •
# .... frame end
# .... infra done.
[0 1 2 3] •
# .. frame end
# .. range_to_zero done.
[0 1 2 3] •
# frame end
-> [0 1 2 3]
joy?
> I really love the power of Factor. It is probably the most powerful language I have ever come across.
What does that really mean? The power to compute any function? Or expressive power?
If you mean the former, it is certainly no powerful than any other Turing-complete language. Anything that can compute primitive recursive functions is as powerful as anything else (though not requiring the same time and space resources).
If you mean expressive power, then you seem to be saying that "the most expressively powerful language I have ever seen takes away my ability to focus on the problem, and spend my brain power remembering what is on the stack where".
Even decently well-structured assembly language doesn't have this problem! You do the subroutine frame linkage dance on entry and exit into your function, and then things are nicely at fixed offsets which can be given symbolic names.
What does that really mean? The power to compute any function? Or expressive power?
If you mean the former, it is certainly no powerful than any other Turing-complete language. Anything that can compute primitive recursive functions is as powerful as anything else (though not requiring the same time and space resources).
If you mean expressive power, then you seem to be saying that "the most expressively powerful language I have ever seen takes away my ability to focus on the problem, and spend my brain power remembering what is on the stack where".
Even decently well-structured assembly language doesn't have this problem! You do the subroutine frame linkage dance on entry and exit into your function, and then things are nicely at fixed offsets which can be given symbolic names.
FUEL, the Factor mode for Emacs, allows you to do almost exactly this: it shows you the stack effect of any given word when you're hovering over it, and you can ask what the cumulative stack effect is at any given point. The only thing missing here would be providing names to what's on the stack, but you can do that by using the lexical vocabulary along with the above.
> I think the problem can be solved though if the development environment shows the stack at each point in time; that way the computer does the thinking instead of the programmer.
If you're up for it I have some code that I wrote with this very idea in mind, in Factor. I'll put it on github later today and put the link here anyway. I agree with you 100%. If Factor were transparent that way and interactive that way (to the point it's almost like a dialog, which it already almost is with its Listener but you and I want stack transparency) then it would probably be one of the most productive languages for fast prototyping (at least).
If you're up for it I have some code that I wrote with this very idea in mind, in Factor. I'll put it on github later today and put the link here anyway. I agree with you 100%. If Factor were transparent that way and interactive that way (to the point it's almost like a dialog, which it already almost is with its Listener but you and I want stack transparency) then it would probably be one of the most productive languages for fast prototyping (at least).
I'd be curious to take a look.
I have plans to do something similar but in the browser in Javascript. Basically, build up the AST based on the commands that are issued against the stack. Then use the Mozilla Parser API to generate Javascript code.
Additionally, with the example arguments supplied you have test driven develop built right in. Not only do you get tests, but it helps you visualize things as you program.
I have plans to do something similar but in the browser in Javascript. Basically, build up the AST based on the commands that are issued against the stack. Then use the Mozilla Parser API to generate Javascript code.
Additionally, with the example arguments supplied you have test driven develop built right in. Not only do you get tests, but it helps you visualize things as you program.
Sorry for the delay: https://gist.github.com/anonymous/6ec7128c10e9ea1f63ad
I agree with you 110%, in fact I can't understand why this hasn't been done yet.
I agree with you 110%, in fact I can't understand why this hasn't been done yet.
I used to use Factor as my main programming language for a few years and really enjoyed the language and environment. It feels like a mix of Forth, Lisp and Smalltalk. I wrote a number of blog posts about it which might help give a feel for what can be done in the language:
http://bluishcoder.co.nz/tags/factor/
http://bluishcoder.co.nz/tags/factor/
Chris, amazing body of work at your blog. What are your thoughts on the continued maintenance factor and long term viability? what caused you to drop out?
I want to like Factor. I've tried to use it a couple of times, and found at least under OSX that the samples I tried to compile (tetris and one other) didn't actually function.
More relevantly, I feel like Factor has lost the literal relationship between machine and software that is Forth's distinctive contribution. Our processors have drifted away from that mapping as well, though there are of course GreenArrays and a few other stack chips out there.
For a Forth that's pleasant to use and retains that feel, I suggest Retro. The Ngaro virtual machine is pure art, and the additions, particularly the [ and ] quotation words, are thoughtful contributions to the Forth lineage.
More relevantly, I feel like Factor has lost the literal relationship between machine and software that is Forth's distinctive contribution. Our processors have drifted away from that mapping as well, though there are of course GreenArrays and a few other stack chips out there.
For a Forth that's pleasant to use and retains that feel, I suggest Retro. The Ngaro virtual machine is pure art, and the additions, particularly the [ and ] quotation words, are thoughtful contributions to the Forth lineage.
Thank you for the pointer to retro[1] and related[2] projects!
I also sort of found Factor interesting -- but much of the point with Forth, is a high-ish level of abstraction on bare metal hardware. If you're going to be using a vm etc... why not program in something that is much more abstract, like Smalltalk, Lisp, Python -- whatever really? Retro I can get, apparently it is small and very portable -- that makes sense (perhaps not more sense than Smalltalk, though).
[1] http://www.forthworks.com/retro/
[2] http://www.forthworks.com/parable/
http://retroforth.org/docs/The_Ngaro_Virtual_Machine.html
I also sort of found Factor interesting -- but much of the point with Forth, is a high-ish level of abstraction on bare metal hardware. If you're going to be using a vm etc... why not program in something that is much more abstract, like Smalltalk, Lisp, Python -- whatever really? Retro I can get, apparently it is small and very portable -- that makes sense (perhaps not more sense than Smalltalk, though).
[1] http://www.forthworks.com/retro/
[2] http://www.forthworks.com/parable/
http://retroforth.org/docs/The_Ngaro_Virtual_Machine.html
Thank you for the pointer. I enjoyed Forth long time ago and wanted to rekindle that love with Factor, but honestly I ran into bumps. I am not giving up, but will definitely check Retro.
IIUC the creator, which had already written a popular text editor (jEdit), and now a successful programming language, considered his objectives complete with Factor and moved on.
AFAIK he still works at Google, and never posted in his blog again (http://factor-language.blogspot.com/).
AFAIK he still works at Google, and never posted in his blog again (http://factor-language.blogspot.com/).
I think the current rumor is that he's now doing lots of drugs and working full time on bcache.
I did not know about the bcache thing - interesting!
Slava created an interesting language but I never saw an explaination of why he moved away from Factor. He just stopped (writing and caring ?) about it.
It took me a while to find at least some explanation of what happened:
http://article.gmane.org/gmane.comp.lang.factor.general/4940
http://article.gmane.org/gmane.comp.lang.factor.general/4505
Still, it's so strange to me: not only Slava disappeared, but most of the original authors stopped participating in the development and even stopped blogging about Factor. See http://planet.factorcode.org - most blogs have been abandoned for years, some of them are dead links... Also, I have not found any public announcement about who will maintain Factor in the future.
I really like what I know about Factor, and I have tremendous respect for Slava and others who created this language. I looked into the code a little bit, and it looks a very well-thought and very serious work, and I have no idea how they could create it in such a short time.
Still, you can't manage a project like this. No matter how great the language is technically, I won't start using it for any serious work if I don't see any vision, any chance that the project will not get stuck in this stage. I really hope that Factor will overcome these challenges. It would be so sad to see such a brilliant piece of work to be abandoned.
http://article.gmane.org/gmane.comp.lang.factor.general/4940
http://article.gmane.org/gmane.comp.lang.factor.general/4505
Still, it's so strange to me: not only Slava disappeared, but most of the original authors stopped participating in the development and even stopped blogging about Factor. See http://planet.factorcode.org - most blogs have been abandoned for years, some of them are dead links... Also, I have not found any public announcement about who will maintain Factor in the future.
I really like what I know about Factor, and I have tremendous respect for Slava and others who created this language. I looked into the code a little bit, and it looks a very well-thought and very serious work, and I have no idea how they could create it in such a short time.
Still, you can't manage a project like this. No matter how great the language is technically, I won't start using it for any serious work if I don't see any vision, any chance that the project will not get stuck in this stage. I really hope that Factor will overcome these challenges. It would be so sad to see such a brilliant piece of work to be abandoned.
I wonder what makes them say this in the FAQ: "a flexible enough type system for concatenative languages has not yet been designed. However, Factor 2.0 may include optional static typing, if a suitable type system can be found." http://kittenlang.org/ gives the impression of having a fairly sensible static type system.
Hey, I’m the author of Kitten. The site & repo are a bit out of date, but I’m currently working on a release. Would be happy to answer any questions. :)
I think the problem is that you want some words with dynamic/value-dependent stack effects. It’s hard to intermingle static and dynamic typing well, and hard to bring the utility of dependent typing for enforcing program properties into the realm of properties that are actually useful to enforce while remaining usable.
I think the problem is that you want some words with dynamic/value-dependent stack effects. It’s hard to intermingle static and dynamic typing well, and hard to bring the utility of dependent typing for enforcing program properties into the realm of properties that are actually useful to enforce while remaining usable.
I've always wanted to learn a Lisp, but I've found Scheme too academic, so I ended up learning other variants such as Common Lisp and Clojure. Could the same thing be said about Factor vs Forth?
Yes. Factor definitely has a much more modern feel but loses a lot of the purity. That said, Factor is still pretty "pure" by the standards of most languages.
Forthish languages are very interesting. I feel they might even represent a "third way" from the normal LISP / FORTRAN dichotomy.
Forthish languages are very interesting. I feel they might even represent a "third way" from the normal LISP / FORTRAN dichotomy.
You get Lisp or Fortran by adding discipline and using the stack language as a back end. Then instead of "pop three things off the stack and put one back", that stuff is hidden and generated by the compiler, so (on the plus side) the stack pointer is never wrong. You don't have to worry about: if I take this branch of the conditional, will the stack be left at a different height. Or worse: be tempted to exploit that somehow as a feature. The stack basically disappears from view. And then (on the minus side) you can't do things any more like replace a function that pops three things, by a sequence of two functions where each pops two and puts one back. On the plus side again, because the stack has disappeared, you can really change how it is represented. Like pass and return in registers; aggressively inline functions, etc.
There's a always been more than just those. E.g. declarative languages, such as prolog.
Yes, and even "FORTRAN" side represent several approaches - for example, straightforward imperative, like in assemblers or classic BASIC, or structural - which was a big step forward at the time - like Pascal.
Then you have object-oriented languages, which are still different from structural, string-oriented Tcl, array-oriented APL...
Then you have object-oriented languages, which are still different from structural, string-oriented Tcl, array-oriented APL...
Sort of. The big difference is that forth is still reasonably widely used. Much more so than factor, not that they target the same domain.
It's entirely my own failing, but I just can't get the hang of postfix languages.
Feels too much like coding in Yodaspeak.
Feels too much like coding in Yodaspeak.
Firstly, I think that some problems just do not map well to the concatenative space. E.g., complex mathematical expressions are a total bitch to try to do entirely with stack shuffling and whatnot. Thankfully, the Factor team agrees, so Factor has a full lexical variable vocabulary that performs just as well as the concatenative variety, and which is used even in the stdlib to do exactly this kind of thing.
Second, there's a mental hurdle to overcome to fight the Yoda speak, which is making sure that all of your words (i.e., functions) are tiny and well-factored. In most languages, having functions that run ten lines isn't a big deal. In Factor, that's a major problem I'd reject in code review. My best functions only run one line. Two is tolerable. In very specific circumstances, if it really makes sense, I'll bust out lexical variables and allow an extra line or two. Anything longer than that means I seriously messed up.
This makes a huge difference, because the thing about having such short functions is that the Yoda speak isn't any worse than when you're doing a builder pattern in Java or the like. There isn't really that big a difference between
Second, there's a mental hurdle to overcome to fight the Yoda speak, which is making sure that all of your words (i.e., functions) are tiny and well-factored. In most languages, having functions that run ten lines isn't a big deal. In Factor, that's a major problem I'd reject in code review. My best functions only run one line. Two is tolerable. In very specific circumstances, if it really makes sense, I'll bust out lexical variables and allow an extra line or two. Anything longer than that means I seriously messed up.
This makes a huge difference, because the thing about having such short functions is that the Yoda speak isn't any worse than when you're doing a builder pattern in Java or the like. There isn't really that big a difference between
foo.Where(x => x > 0).Select(x -> Math.pow(x, 2));
(->> foo (filter positive?) (map #(^ _ 2)))
foo [ > 0 ] filter [ 2 pow ] map
Some short functions look so beautiful... but I have a hard time as stuff gets more complex.
https://github.com/slavapestov/factor/blob/master/extra/anag...
IMO this:
https://github.com/slavapestov/factor/blob/master/extra/anag...
IMO this:
MEMO: dict-words ( -- seq )
"/usr/share/dict/words" ascii file-lines [ >lower ] map ;
I believe the same is true of other paradigms. Of course complexity is difficult. Do concatenative languages exaggerate the effect?
I think so sir - I had only a couple beers and can't parse the function below.
: <color-picker> ( -- gadget )
vertical <track>
{ 5 5 } >>gap
<color-sliders>
[ f track-add ]
[
[ <color-model> <color-preview> 1 track-add ]
[
[
vtruncate v>integer
first3 3dup "%d %d %d #%02x%02x%02x" sprintf
] <arrow> <label-control>
f track-add
] bi
] bi* ;It helps to be familiar with the words (Factor term for functions) in use. You can use http://docs.factorcode.org to look them up. All the help is also available from within Factor itself. You can click on word definitions and it comes up.
In this case there is a word called '<color-picker>' being defined. It takes no arguments off the stack and leaves one, gadget, on it.
Now working from left to right, 'vertical' is a constant. '<track>' creates a new GUI object called a track that requires the orientation to be on the stack. That what 'vertical' is. It's the orientation.
Now that there is a track object there we fill in the slots of that object. That's what '>>foo' does. It stores something in the 'foo' slot of the object on the stack. So "{ 5 5 } >>gap" stores the 2 element vector containing the integer 5 twice, into the 'gap' slot.
'<color-sliders>' creates a gadget and a model that is left on the stack. This is what 'bi-star' is now operating on.
'bi-star' is a combinator. It requires two objects and two quotations (quotations are factors term for anonymous functions or closures). It applies the first quotation to the first object, and the second to the second object. This system of combinators is described in the Joy language writings, although I think under different names.
"[ ... ]" is the syntax for the quotation. So "[ f track-add ]" is the first quotation. "bi-star" call this with the 'track' on the stack since that's the first object that "bi-star" gets access to.
The rest of the code can be worked through in a similar manner. In general you learn to recognise things like "<...>" for creating objects, ">>foo" for setting slots, "[ ... ]" for quotations and combinators to reduce stack management (like dup, rot, pick, etc).
In this case there is a word called '<color-picker>' being defined. It takes no arguments off the stack and leaves one, gadget, on it.
Now working from left to right, 'vertical' is a constant. '<track>' creates a new GUI object called a track that requires the orientation to be on the stack. That what 'vertical' is. It's the orientation.
Now that there is a track object there we fill in the slots of that object. That's what '>>foo' does. It stores something in the 'foo' slot of the object on the stack. So "{ 5 5 } >>gap" stores the 2 element vector containing the integer 5 twice, into the 'gap' slot.
'<color-sliders>' creates a gadget and a model that is left on the stack. This is what 'bi-star' is now operating on.
'bi-star' is a combinator. It requires two objects and two quotations (quotations are factors term for anonymous functions or closures). It applies the first quotation to the first object, and the second to the second object. This system of combinators is described in the Joy language writings, although I think under different names.
"[ ... ]" is the syntax for the quotation. So "[ f track-add ]" is the first quotation. "bi-star" call this with the 'track' on the stack since that's the first object that "bi-star" gets access to.
The rest of the code can be worked through in a similar manner. In general you learn to recognise things like "<...>" for creating objects, ">>foo" for setting slots, "[ ... ]" for quotations and combinators to reduce stack management (like dup, rot, pick, etc).
Are you experienced with the language?
I can't parse it either, but I don't know if "readability for a beginner" is a metric I care a whole lot about in a programming language.
I can't parse it either, but I don't know if "readability for a beginner" is a metric I care a whole lot about in a programming language.
They do in my (marginal) experience. I have found it to be a very common idiom to keep word definitions in these languages as short as possible. Probably for this very reason.
You define new words for the sole purpose of keeping things short and to the point. This holds true for most languages (At least if you want readable and maintainable code), but a language like Factor, this becomes an absolute necessity.
I would not be surprised if this is the reason Factor is called Factor. Specifically when related to factoring in Algebra where you do much the same thing: "splitting" an expression into a multiplication of simpler expressions.
You define new words for the sole purpose of keeping things short and to the point. This holds true for most languages (At least if you want readable and maintainable code), but a language like Factor, this becomes an absolute necessity.
I would not be surprised if this is the reason Factor is called Factor. Specifically when related to factoring in Algebra where you do much the same thing: "splitting" an expression into a multiplication of simpler expressions.
The Postscript language might be another chance for you to flex you postfix muscles. Still concatenative, dynamic programming, with built-in graphics.
Don Lancaster[1] is notable for producing a long running newsletter with a library of hand coded Postscript to handle layout and rendering of charts and graphs. He provides all the source code for those interested.
[1] http://www.tinaja.com/gurgrm01.shtml
[1] http://www.tinaja.com/gurgrm01.shtml
Maybe of interest: Zed Shaw, creator of Mongrel, gave a talk a few years back where he mentioned that he was using Factor to create the slides in his presentation.
http://vimeo.com/2723800
http://vimeo.com/2723800
Sample "presentation"
https://github.com/littledan/Factor/blob/master/extra/galois...
One thing I find interesting about stack languages is that if you imagine the stack tilted by 90 degrees, you realize that you are doing operations in some monoidal category.
Maybe of interest too : oforth language
That being said, I think there needs to be a more interactive method where you can see things on the stack in real time as you are programming.
I found myself not being able to focus on the problem I wanted to solve and instead was spending too much of my brainpower on remembering what was on the stack where.
If you study human computer interaction and cognetics the human brain can only remember so many short term memory "variables" at once. Programming in Factor uses up too many of those spaces.
I think the problem can be solved though if the development environment shows the stack at each point in time; that way the computer does the thinking instead of the programmer.
One way to do this is to supply some arguments to the word (function) and as you manipulate it in real time you can see how it is transformed. Your steps get recorded as the word.
This solves the problem of programming blind and having to play interpreter in your head as you are programming. Thus, freeing up cognitive space and making the programmer more efficient.