This is not true. Both OpenAI and Google's LLM APIs have a policy of not using the data sent over them. Its no different than trusting Microsoft's or Google's cloud to store private data.
The Aider leaderboards seem like a good practical test of coding usefulness: https://aider.chat/docs/leaderboards/. I haven't tried Cursor personally but I am finding Aider with Sonnet more useful that Github Copilot and its nice to be able to pick any model API. Eventually even a local model may be viable. This new Gemini model does not rank very high unfortunately.
> someone who doesn't appear to respect the organizational capabilities of academic labs, you are condemning them to far more poorly organized outputs.
This is not a great way to make your argument, though you are not the not only one here making a personal judgement without even knowing about my background. These are all issues I have seen first hard. With most academic labs being funding limited, the "organizational capabilities of academic labs" seems irrelevant to me. In our field, no one is getting grants to manage code of any kind .py or .ipynb and I suspect its the same at most university labs. It's effort wasted that ultimately does take time away from the actual research that's fundable and publishable. As someone who has been responsible for wrangling people's notebooks in the past, it's enough of a problem that I would encourage to remove all .ipynb.
> That doesn't have anything to do with notebooks. It's as silly as saying that a Python package is a poor idea because you say somebody repeat code across multiple places.
Human factors make jupyter notebooks lead to the problems I have listed. The issues are most apparent with large groups and over long periods of time. Python and other programming languages already solved most of these problems with git. There isn't a tool that is as elegant and scales from individuals to massive organizations.
> There are roughly two modes for notebooks: exploration with a REPL, and well-documented reports. The best scientific reports I have ever seen are notebooks (or R Markdown output) that are the full report text plus code plus figures.
The REPL functionality is handled by .py cell execution, as I’ve mentioned in other comments. It baffles me how the minimal effort saved by not using separate tools -- one for code, one for documentation -- justifies the issues it introduces.
Yeah, jupyter notebooks don't guarantee any specifics about versions of code used for that output. In the real world you can expect everyone in the lab including all of the students to be editing jupyter notebooks at whim. The only way to do this would be to have proper version control and of your code, a snapshot of the environment, and to log all this along with the run that generated the output. This is possible with regular python using git, proper log files, etc. Jupyter notebooks seem like an extra roadblock.
I've looked into Jupytext, but ultimately decided to go with pure python. Most of the practical functionality can be replicated, but I do admit there isn't a easy single install tool or guide to replace notebooks at the moment.
I think the notebooks are a fine learning tool to introduce people to programming initially, but I'm afraid it doesn't allow for growth beyond a certain level. You have a good point about funding for those software roles. Perhaps this may not be as big of a concern if there were more software talent in these labs to handle the issues that arise.
That seems like a reasonable way to use jupyter notebooks since you have an actual plan to move beyond it when necessary. My issue is mostly with the way its misused, often by people who are arguably at the top of the field.
Not having the outputs tied into the code is actually preferable if the ultimate goal is reproducible science. Code should be code, documentation should be documentation, and outputs should be outputs. Having multiple copies of important code in non-version controlled files is not a good practice. Having documentation dispersed with questionable organization in unsearchable files is not good a practice. Having outputs without run information and timestamps is not a good practice. Its easy to fall in to those traps with Jupyter notebooks. It might speed up initial set up and experimentation, but I've been working academic labs long enough to see the downstream effects.
Imo, the better architected .ipynb is simply .py with '# %%' blocks. It does almost everything a .ipynb can do with the right VSCode extensions. Even interactive visualizations can be sent to a browser window or saved to disk with plotly. Though I do wish '# %%' cell based execution was accessible to more people.
There isn't a single install tool that "just works" for this at the moment. If editors came with more robust support for it by default, I think the notebook format wouldn't be needed at that point and people could use regular python and interactive cell based python more interchangeably. I've seen important code get buried under collections of jupyter notebooks across different users so I have a good reason for this. Notebooks simply dont scale beyond a certain complexity.
I dislike how Jupyter notebooks have become normalized. Yes, the interactive execution and visuals are nice for more academic workflows where the priority is quick results over code organization. However, when it comes to sharing code with others for the sake of doing reproducible science, jupyter notebooks cause more trouble than they are worth. Using cell based execution with python is so elegant with '# %%' lines in regular .py files (though it requires using VSCode or fiddling with vim plugins which not all scientists want to do I suppose). No .ipynb is necessary, .py files can be version controlled and shared like normal code while sill retaining the ability to use interactively, cell by cell.
Its much easier to organize .py files into a proper python module, and then share and collaborate with others. Instead, groups will collect jumbles of slightly different versions of the same jupyter notebooks that progressively become more complex and less manageable over time. It's not a hypothetical unfortunately, I've seen this happen at major university labs. I'm not blaming anyone because I understand -- the funding is there to do science and not rewrite code to build convenient software libraries. Yet, I can't help but wish jupyter notebooks could be removed from academic workflows.
Mojo claims to be interoptable with python, but it seems only one way. There's no built in way to use the binaries and use them with a regular python environment. It would be hard to incrementally migrate from an existing project. Something like Cython would allow this. Couldn't the GPU features be accessed from C/Cython instead? Migrating a whole project to Mojo would require more time investment and I am not even sure if all the dependencies would be compatible.
You are probably right, but that is truly a cyberpunk dystopian situation. A few megacorps will catalog every human interaction and there will be no way to opt out.
Its certainly not altruism. Given that Facebook/Meta owns the largest user data collection systems, any advancement in AI ultimately strengthens their business model (which is still mostly collecting private user data, amassing large user datasets, and selling targeting ads).
There is a demo video that shows a user wearing a Quest VR headset and asks the AI "what do you see" and it interprets everything around it. Then, "what goes well with these shorts"... You can see where this is going. Wearing headsets with AIs monitoring everything the users see and collecting even more data is becoming normalized. Imagine the private data harvesting capabilities of the internet but anywhere in the physical world. People need not even choose to wear a Meta headset, simply passing a user with a Meta headset in public will be enough to have private data collected. This will be the inevitable result of vision models improvements integrated into mobile VR/AR headsets.
I am aware people use it, but its only a partial solution that introduces new issues. For certain users it may be sufficient but even if you ignore the latency and compression issues, being unable to have multiple headsets adjacent makes it an incomplete solution if the goal is spreading the technology to new users. Even multiple WAPs won't get around the bandwidth congestion.
There's several problems with that. 1. The latency - even if fine for slower games its enough to cause slight vestibular-ocular mismatch and discomfort. 2. Compression and visual quality - not only is the quality worse but its also cost using more GPU resources for the lower quality compared to a DisplayPort signal. 3. the Wifi 6E RF bands and protocols are not suited for lossless, low latency video transmission. Especially as we approach 4K per eye resolutions, the bandwidth is not there and relying on additional encode/decode hardware adds more latency and artifacts. We also need a solution that would allow for multiple users in a single building to use wireless VR, which I don't believe will be possible with the RF bands available.
There was new Wigig standard that may have solved this, but I believe its not being used anywhere.
I think one of the big problems with VR (and also AR) is that the large companies lack focus and have been trying to make generic do-everything devices to cover many applications. It may make sense from a business point of view since it would ensure the largest user base. However for such a new technology that has a wide variety of potential applications, this means that no one application is given the resources and attention it really needs. Hardware upgrades have negligible impact, software and ux design is not focused on a specific need, and many fundamental issues are left unsolved.
VR companies are trying to make the iPhone of VR without considering that the iPhones success was built on decades of computing fundamentals. Before its possible to make good hardware and XR experiences, we'll need basic research in optics, display panels, tracking, multiview eye tracked foveated rendering, gaze correction, vari-focal, lossless wireless... the list can go on and on. Very few want to invest in solving these problems and simply wants to build a huge ecosystem with a large user base. Even Facebook/Meta, who have invested the most, have failed to tackle any of the major problems even after 10 years of being in the field. Since 2016 when 6-DoF tracked controllers became the norm, there hasn't been any major advancements other than slightly better visuals.
Looking at XR technology that has been successful, its usually because of a very clear focus on a specific application. VR flight and racing simulations with professional headsets like the Varjo appear seem far more developed. With a motion rig, these are good enough for training professionals. VR has solidified its place its this niche market at least. Microsoft's success often gets overlooked, but they have a $20B defense contract to supply IVAS AR headsets to the US military. If more companies focused on solving one of fundamental problems, it should eventually be possible to create a mainstream mass-market device that everyone will want to use.
This is what I do essentially. I make a new conda env for each project and use pip or conda install. What if I have a new project that needs components from two projects? Sometimes there will be impossible to solve dependencies when trying to use both components. Its not feasible to dive into each dependency within each dependency to figure out how to resolve them.
Rust's package manager, cargo, is able to handle this by allowing multiple versions of libraries to be installed in a single environment. Why can't python do that? How can one solve this with conda/pip or any currently available python tool? I've given up and decided to use websockets between different python processes from different environments.
My concern is the immunity allows for perpetual, single party rule with only a 1/3 of senate seats and the presidency. There would be no checks in place at the federal level if the party was willing to exercise power to interfere with elections or opposing candidates.