The UI is implemented using Halogen, a front end library for PureScript. That handles redisplay when recalculation happens. That’s more like React or Elm.
Rather, I mean exposing an FRP API to people using the Inflex language.
I'll be adding this to Inflex (https://inflex.io/), I have the design worked out on paper. (But I'll be open sourcing it first and releasing as a desktop app, and then get back to dev.)
It's also helpful to think of FRP in terms of "push" and "pull" (for which there's a related paper by the same chap). This refers to control flow. Behaviours are "pull" i.e. your program has to pull from them. Events are "push" i.e. your program gets pushed to, by some other active agent.
The trick is how one "makes things happen". If you want to pull the latest tweets every 5 seconds, in reverse order, you might have code that looks like:
timer(5).joinWith(latestTweets).map(reverse)
Similar to promises you're always building up more declarative values. It's just that FRP has a clean semantic description. There are laws, and no assumptions about time/imperative escape hatches. It can be quite hard to use it practically for some types of apps; space leaks and cycles are a challenge, and some code can be messy. Research continues.
But for my simpler use-case, it's a very good fit. Spreadsheets use "volatile" cells to side-step the whole issue of interacting with time and the outside world, but it leaves a bad taste in the mouth because it's a hack that makes state implicit. FRP brings a strong mental framework to address this properly, rather than as an afterthought.
> There is a secret that needs to be understood in order to write good software documentation: there isn’t one thing called documentation, there are four.
>
> They are: tutorials, how-to guides, technical reference and explanation. They represent four different purposes or functions, and require four different approaches to their creation. Understanding the implications of this will help improve most documentation - often immensely.
Thanks for sharing. I just realized your use case could fit quite neatly into Inflex (my app, not open for signups, but has a “try” sandbox), a use case I never considered.
I have an unexposed WIP type of cell that is a rich text document. The rich document you edit has corresponding code, which you can also edit in the other direction. (screenshot https://mobile.twitter.com/InflexHQ/status/14923564133263360...) The neat part is that I added the ability to embed a “portal” to display another cell, such as a number or a table, which can be edited inside the rich text doc or at the cell’s origin. For tutorials, I figured it would also be nice to display the source code of a given cell in a rich editor. E.g.
The formula [plasma * 3.23224 + alpha] yields [7.289221].
You could edit the text (The Formula ... yields ...), or the formula and see the result.
Finally, the rich editor is already in a format ready to be printed as a PDF or Word doc.
Also, the source code being the source of truth means it’s very easy to version the whole system down to SHA512 hashes.
This could unify a use-case like yours, where you have the Google Doc and the Google Sheet bridged by Python, this would cut down that iteration feedback loop.
Being content addressable means that only tests that need to run would be run (see Unison lang), rather than running all tests every time, further cutting down on the feedback loop.
One question that comes to mind is: what if you could export code in the spreadsheet as a general purpose language like C or Python? Or even Lua? Also, would an on-prem/desktop version of the product would be valuable for this use case?
Thanks for the food for thought. This is a niche I never considered! And I’ve worked on a medical device for Amgen before! I forgot all about this.
In Inflex decimals are not floating point, they’re fixed precision. So 0.1+0.2=0.3 produces #true. You can try by hitting the Formula button here: https://inflex.io/try
As for the problem of which things of a data frame are generated from a formula and which are from normal form, Inflex compares the structure with the AST of the source, making it easy to tell that [{foo: ...}] are normal form and so editable graphically, whereas xs.filter(..) is not. The neat thing is you can still edit formulae that are deep within a normal form nested structure.
It helps that Inflex doesn’t have syntactic sugar, what’s parsed is what is in the final AST. It also has a symbolic evaluator, somewhat, so it’s fine to have a list of functions for example and edit the list. The evaluator produces the same AST, rather than an alien format.
Mito has the advantage of being a familiar language and ecosystem, but Python itself has a traditional runtime, it’s imperative and not expression oriented, and lacks sound static type information, so it’s inherently more difficult to achieve some things with it that are easier with a typed pure functional language, especially a custom one.
I realized I left that open ended but I’ll touch on a few points that I think are relevant here;
* There’s a language which is like Haskell/Elm/PureScript in terms of being purely functional and statically typed. But with syntax that looks more like Excel.
* Purity gets us fearless recalculation.
* Static types let us build UI elements automatically based on the inferred types of code.
* It’s content-addressable like Unison. That means every expression and “cell” has a unique SHA512 hash of it which refers to only that expression.
* Content addressability makes cache invalidation of results trivial.
* It also makes it easy to say “I want exactly this version of that person’s cell and for all time.” Makes it impossible to break someone else’s code once it’s working.
* It also lets you fearlessly federate, if ever needed.
* Content addressed also means you can write tests against code and have them run on every change. Only the tests whose dependencies changed will be rerun. That’s not normal in Python or Haskell, but in a spreadsheet it is.
There are other design choices related to your comment but I don’t want to ramble on.
I would add that any tool looking to replace Excel will be build on some very powerful but very primitive foundations, I’m order to compete with that flexibility. It’ll never be about adding a special “view” like AirTable or just tacking in Python.
Excel is like Emacs; most users will write some Elisp at some point, it’s designed to be meddled with from the ground up. AirTable is the VSCode; most users will never write a line of plugin code and when you do you’ll find you can’t extend much.
I’m trying my best to create such a product. What you’re describing is something that (1) amateur spreadsheeters can use, (2) the power users, and (3) programmers, will be able to use and not feel out of their depth or patronized. I personally think that given that Excel is a pure-ish declarative language, then building on a declarative language that programmers use is a solid attack on this, which is why I’ve chosen pure functional programming as a basis for my work. It’s been an enjoyable design process to include or throw out ideas that would alienate either end of the spectrum.
Elsewhere: Making all cells CAS gives you a strong foundation for version control, modularity, resizability and share-ability.
Happy to share more thoughts on this. I’m two years into the process.
My reactive workspace tool aka spreadsheet replacement (Inflex) is SaaS, but I’ve also considered offering a completely downloadable tool for a one off price. I think if ever get a substantial user base and people ask for that, it’s on the table.
> Another thought along these lines: quite a lot of my spreadsheet use involves doing calculations on whole columns. This is really painful in Excel, where I have to manually select just the right areas of my sheet, then write the formula—without disturbing my selections!—making sure of course that I put the result in a place where it fits. And this just gets worse with more data. In my own proof-of-concept, I implement broadcasting of all functions over arrays (à la APL or MATLAB), which makes this a lot easier, and is quite possibly the best UX improvement I’ve found. Do you have any plans to do something like this in Inflex? (I know it already has higher-order functions, but e.g. `voltage / current` is a lot easier to read and write than `zipwith(x y:x/y, voltage, current)`…)
I haven’t seriously put thought into that use-case. There’s the facility via type classes, but I’m not sure what the trade offs are with that kind of overloading. The same question applies to FRP behaviors and events. Do you explicitly require a map or list comprehensions or do you “broadcast” automatically, and if you do, what are the pitfalls?
I have put some thought into something related which is aggregations. On a given table I want to be able to add a column which implements any mapAccum over the table, such as balance over time. The code generated would be table.accum(state: row: state + row.x) for example. The UI would make it easy to pick common accumulators and folds, but also trivial to write an expression. Your voltage current example would work like that.
This arrays-of-records overlaps a bit with your records-of-arrays naturally. The two are often considered sides of the same coin and I’ve thought about using RoA internally for efficiency, but “broadcasting” to make that nice didn’t occur to me. The written simplicity is compelling, for sure.
> From my perspective, they’re definitely worth it: most of my biggest Excel mistakes have involved botched unit conversions. (If it matters, my background is physics and chemistry; most of my spreadsheet usage has been for assignments throughout my degree.)
Yeah, it’s definitely a valuable thing, I’m just not sure about the trade offs. At some point you have to stop adding features to the language, and that one didn’t convince me enough at the time.
> applying substitutions at the same time as unification
Right, I’ve seen that also on a language we develop at work. Incidentally this is needed if you want to combine two row types and assign a type to the expression.
> If it helps, I’d be more than happy to help out with development! This is an area I find incredibly interesting to work on, but my own attempt never really got past the proof-of-concept stage, and I consider duplicate development a bit of a waste.
I might only have time to discuss ideas and no time to collaborate on dev right now, but happy to swap contact details. If you drop a mail [email protected] I’ll reply via my personal email.
Inflex is statically typed, it’s pretty much PureScript with more familiar-to-Excel syntax, with row types used for records and also polymorphic variants, which aren’t in PureScript but are in OCaml. Numbers and ordering and comparisons are dealt with via type classes.
You can annotate a cell’s expression with a type signature, though this is not explicitly mentioned or supported intentionally as I’m not decided fully on the syntax. Example: when you make a “table” via the Table button, it just makes a cell whose source code is: [] :: [{"column1":_, "column2":_}] as tables are just lists of records. If you go to https://inflex.io/try (work not saved, this page doesn’t hit the DB whatsoever) you can hit Formula to write code. But all cells are code underneath (hit the triangle).
My next addition to Inflex will be push-pull based FRP, for dealing with time, buttons and external events and outputs. I also plan on having it scale so that cells with large tables are refined into real database rows for more efficient operations. So the goal of Inflex is to embrace the small scale (lists and easy spreadsheet stuff) with a smooth migration path towards more advanced programming, in a coherent whole. But you have to start with the simple and work your way to the fancy stuff, or so I think.
I’ve considered units of measure, but they are complicated from an end-user perspective and complicate the type system, I’m not sure whether they’re worth it over providing very good automated property based testing. That’s an ongoing consideration.
Development is slow in my spare time which is limited, but I’m comfortable going at my own pace. I’d like to roll out a discourse forum to document things, but hosting is quite expensive. It’s worth resolving this, though, because I’m doing a poor job at explaining the product. One is always choosing between adding more polish/features and documenting!
This in principle accurate, but the real story is less polished. If you look through the AirTable forums, you’ll see lots of cases where users are asking how to do a thing, and are told to abandon the highly limited scripting and to use JavaScript. This is what I characterize as a learning cliff.
It does have limited calc, but I tend to think of it as more of an afterthought. The scripting is “very incomplete” in the words of the author, consider e.g. https://github.com/aardappel/treesheets/issues/37
I’ve also seen that there’s a whole sub industry of companies that build a platform on top of a spreadsheet by assuming a certain structure of the data. Aside from myself, Julia’s Pluto.jl, ObservableHQ, and a few researchers, there aren’t many trying to invent a new spreadsheet replacement with a fundamentally different conceptual model.
I think Excel fails when it comes to other natural types of data such as trees, graphs, even arrays or records. It’s terrible at all of them. It’s not relational, and it’s not inductive, has no sum types, or anything.
What it is, is a very good UI to throw tabular data into and modify.