Nip2: A spreadsheet-like GUI for the libvips image processing library(github.com)

97 pointsby valgazevor 4 Jahren23 comments

Nip2: A spreadsheet-like GUI for the libvips image processing library

https://github.com/libvips/nip2

23 comments

erdmannvor 4 Jahren

I would also like to sing the praises of nip2. I use it constantly in my work with enormous images at the Rijksmuseum. It's backed by libvips, which means that it offers a delightful combination of insane speed and unlimited image size. The finishing touches of the 717 gigapixel image of the Night Watch were done in nip2. Also, the spreadsheet-like nature of the interface is excellent for designing, documenting, and repeating complex workflows. Cells contain images, and the reactive trickle-down computation lets you build up multi-step transformations and computations cell by cell. Any cell's contents can be replaced by a new image at will, so it's very easy to repeat an analysis or transformation for new source images.

John Cupitt (jcupitt here), the main developer of nip2 and libvips, is super helpful, responsive, generous, and patient.

They can pry nip2 from my cold, dead hands.

kawspervor 4 Jahren

I've reported many strange bugs to jcupitt and I can speak to his really helpful and patient nature.

My favorite one was when JPEG2000s that had an even width was being shown in greyscale on any mac app: Preview, Safari, Pixelmator, etc: https://github.com/libvips/ruby-vips/issues/345

All images with an odd width was working fine.

However the real bug was to do with chroma subsampling (and a lot of other technical stuff that I don't really understand): https://github.com/libvips/libvips/issues/2965

jcupittvor 4 Jahren

Thank you!

(the money is in the envelope on your desk)

Cyberdogvor 4 Jahren

GitHub says that 32.3% of the codebase is in "Witcher Script." What the heck is that?

> Witcher Script (.ws) is the primary scripting language for The Witcher 3: Wild Hunt. A very large chunk of the game's logic is written in Witcher Script. Mods can override scripts and thus can drastically change game behavior. Witcher Script is either based on or identical to ActionScript 3 (and very similar to UnrealScript).

https://witcher-games.fandom.com/wiki/Witcher_Script

Interesting. CDPR had a NIH problem. At any rate I'm guessing GitHub is misidentifying something.

dljsjrvor 4 Jahren

If you hit `T` to open the file searcher and look for `.ws` it brings up a bunch of "Workspace" XML files. So it's misidentifying a config file.

jcupittvor 4 Jahren

nip2 has `.ws` files for storing workspaces (just a big chunk of XML), and `.def` files for its own scripting language.

The nip2 scripting language is also very odd: it's something like dynamically typed Haskell, but with classes. You can see roughly what it looks like here:

https://github.com/libvips/nip2/blob/master/share/nip2/start...

All of the menus are implemented in this thing.

electrolyvor 4 Jahren

You can use a linguist directive in your .gitattributes file if you're interested in fixing GitHub's language identification for those files. A line like the following would hide these files from GitHub's language meter bar.

  *.ws linguist-documentation

jcupittvor 4 Jahren

Hi electroly, thanks for the tip!

game-of-throwsvor 4 Jahren

> CDPR had a NIH problem.

Bit of a tangent, but I don't think that's true. It's just ActionScript with a cute file extension - just like .docx is .xml with a cute file extension.

makeworldvor 4 Jahren

Reminds me a bit of GRIP. Unfortunately there seems to be no screenshots, but the flow/modular interface is similar.

https://wpiroboticsprojects.github.io/GRIP/#/

jcupittvor 4 Jahren

Yes, I based it partly on visual programming system that Canon made back in the 90s. I loved the interactive development, but I really disliked that style of visual programming. Lines between boxes seems very hard to reason about, to me.

nip2 tries to do something similar, but using expressions, like a spreadsheet. Unlike graphical plugs and wires, expressions allow things like copy-paste, smart refactoring, referential transparency, and progressive rewriting.

nip2 is ancient now (I finished it way back in 2002, I think). I've started a rewrite for gtk4 -- the image display window is here:

https://github.com/jcupitt/vipsdisp

It's quite fancy -- it has a sparse pyramid of image tiles which get composited to your display by the GPU, and computed asynchronously by a libvips pipeline as you zoom and pan around. You should get a smooth 60 fps even with enormous (eg. 300,000 x 300,000 pixel) images.

dansman805vor 4 Jahren

There are some screenshots of GRIP here https://docs.wpilib.org/en/stable/docs/software/vision-proce...

hawskivor 4 Jahren

This looks nice and useful. I was thinking for a long time about a graphics manipulation program that would put every step as a separate line with all parameters visible. Then you can change any parameter from earlier steps as you see fit. It makes the graphics editing process like a little interactive script. You could then easily parametrize it further and even do batch processing this way.

I will have to check this out.

jcupittvor 4 Jahren

Yes, nip2 has a no-GUI batch mode.

You can make a workspace interactively that does some task, then run it with something like:

nip2 -bp --set 'Workspaces.process.A1=Image_file "fred.jpg"' --set main=Workspaces.process.A10 -o fred2.jpg process.ws

-b meaning batch mode, -p meaning send the value of main to the output. That'll load the process.ws workspace, set cell A1 to be the image "fred.jpg", set main (the point of execution) to be cell A10, and save it to fred2.jpg.

I made a workspace for medical image analysis (my old job). It finds lungs in a 4D PET-CT scan, then fits a two compartment model per voxel and generates a volume image from k2. It has just over 10,000 cells (!!), 70GB of images, and executes in about 2 minutes using around 1GB of ram.

I then used a small bash script to loop over my dataset of 2,000 scans computing an inflammation index. It worked surprisingly well, IMO.

snerblesvor 4 Jahren

This seems similar to Cognex's spreadsheet programming model in their industrial smart cameras.

rsstackvor 4 Jahren

What makes this a spreadsheet rather than a notebook (like Mathematica or Jupyter)? It's still great, I just expected something else - something more... spreadsheety.

jcupittvor 4 Jahren

I suppose it's only a spreadsheet in a very broad sense: a set of formula which refer to each other, and which are updated automatically on any change.

You can't have circular references to the top level, but you can do recursive definitions within a cell, which is maybe spreadsheety too.

You can use scraps of code to set the GUI behaviour, which is a bit crazy. Try making a region on an image (open an image view window, hold down CTRL, drag down and right).

Back in the main window, press the down button to the left of to the label of the cell containing the region and you'll find you can open it up. Keep clicking and a lot of rows will appear -- including left, top, width, height, etc. If you drag the region around and resize it, you'll see all these numbers update.

Click on the number for height and you can edit it. Enter "width * 2" and try resizing the region.

orbital-decayvor 4 Jahren

Spreadsheets are typically 2D tables in the first place, relations are secondary. This looks more like a node graph, which is a widespread UX paradigm in DSP and content creation. Almost every VFX/3D/image processing/sound engineering software suite has a graph editor or several somewhere underneath, or even entirely built around a node-based editor.

jcupittvor 4 Jahren

Yes, nip2 workspaces are DAGs, like node graph interfaces. The spreadsheet comparison was originally there to help introduce the interface to a less technical audience. nip2 was written in the 1990s when this stuff was less widespread.

I don't like plug-and-socket style graphical node graph interfaces personally -- I think they are hard to reason about, and things like composition and rewriting feel unnatural (to me). I came out of the functional programming community and I wanted things like referential transparency.

nip2 is supposed to be a graphical interface that's an expression of a set of relations. It has a three part model:

1. You enter scraps of code (or pick menu items) to build an object in nip2's programming language.

2. nip2 looks at the constructed object and draws the GUI.

3. Changes to the GUI trigger "edit" functions on the constructed object to update it.

4. ... if you go back and edit the code that originally constructed the object, the GUI resets.

So your code and the GUI independently update the constructed object, with code taking priority. You get a stateful GUI with a purely functional programming language and you get to keep equational reasoning.

JodieBenitezvor 4 Jahren

I don't use Nip2 (yet), but since you're here let me express how grateful I am for libvips and its Python bindings. It's a fantastic and essential tool.

jcupittvor 4 Jahren

I'm glad it's useful!

rsstackvor 4 Jahren

I installed nip2 and played with it for a while. It is _the best_ notebook/spreadsheet tool I ever experienced. I don't know if the folks from Equals.app have notifications on their name in Hacker News, but they should consider taking some inspiration from nip2 and not just from Excel.

ArrayBoundCheckvor 4 Jahren

Does it send me notifications? Is it called NipAlert2? Is professor bighetti a collaborator?