I tried to do this with hacker news data [0]. I wanted to feed the model the entire community's discourse and then ask it questions (like simulating an interview with a HN user). The main problems encountered were:
- 1. Token limit: You can only input a limited amount of text at once. The challenge then becomes trying to compress data to fit into the window. But it can be lossy.
- 2. Trust: This is the main one. It's hard to determine if the output is based on the new learning material or the large amounts of data the model was originally trained on. There are techniques that can help but they add a lot of additional work and don't guarantee great results.
That's close to the conclusion I come to in my experiment [0]. Focusing on the generational capabilities can make some cool demos but investing in good search felt like the most useful thing to do.
While the author's anecdotal evidence is coherent, it's not enough to establish causation. Especially if the measure of productivity is feelings of sluggishness.
Rust or not, there's an argument to be made for statically typed languages improving productivity over the long run [0].
All development can feel sluggish depending on the work hours, estimates, business timelines, engineering skills, and the task at hand.
Programming languages are common targets because
- we use them so much
- there are so many
- and all developers, at some point, must choose to dedicate their time to one over the others.
Finally, productivity itself is only one performance characteristic. To focus only on that (without even a good definition of effect or measure) makes content precisely what the author claims to avoid; flame bait.
Separating functions helps us use the right tool for the job. Taxonomies are for semantics, and the file system is for retrievability. The comfort of hierarchies makes it easy to try and do both simultaneously.
- From computer science, we know graphs give us expressive modeling capabilities. I sometimes use mermaid ER diagrams as a concept map to capture complex relationships between files and concepts.
- From library science, faceted classification works well for extensive collections because inserting a new entry does not require thinking about existing entries. I maintain entries in a spreadsheet for extensive collections that matter to me. Note: Facets are meant for unchanging or infrequently changing properties.
Creating a concept map and maintaining a faceted classification system take work, so I only use them for things that are very important to me.
90% of files I only care about for a short amount of time. I use the file system to co-locate the files I'm currently working on (so a project) but then archive all of it when I move on to something else.
The trade-off is that I give up on sharing files between projects. I don't want to deal with references. I copy from the archive when I need to. On the rare occasion when I need to reconcile the same file between projects, I do it manually. What helps is working on only a few projects at the same time.
TL;DR: Archive more. Use high-investment techniques only for the small percentage of files that really matter.
Just thinking about the problem some more (from the perspective of the data owner), I wonder if instead of solving the problem at migration time the problem could be solved at the time of data creation.
What if data was routed to a) the Saas tool and b) to long term storage. So the act of migration becomes an act of moving raw historical data to the new tool instead of pull from service, transform and push to new service.
Never used either while modelling professionally but I think CIM's focus is slightly more on interoperability while Schema.org's goal was on sharing semantics and metadata. Those goals have a lot of overlap and the projects also have common contributors. Not sure why both exist.
My first instinct is to develop a Hotswap unified schema, something like (https://cloudinformationmodel.org/cim-model/) and then develop bidirectional mappings to particular services. Then transformation is a repeatable {Saas pool} -> Unified model -> {Saas pool} transformation. My second instinct is to reconsider the over engineered mess my first instinct came up with. Good job on getting to the point and having things work. Looks like a cool offering, good luck with it!
I've never worked with Clojure and the article does a poor job explaining how it contributes to feature shipping speed. Functional programming? REPL?
I've personally experienced being part of teams that have a "rhythm" and it does make work seem more fun. It usually happens around a big release, after the hard/time-consuming parts are done and the teams are focusing on polishing up the system. I'd be more interested to know how problems are decomposed to support this velocity of features. How do they maintain quality and spend time on refactoring with this pace?
So I signed up for the free tier. There's an option to compose mail that's paylocked. Do I need my own domain for that? If not, i.e if I'm using paced for outbound mail, how are you currently managing reputation management?
Pretty cool service. If in an existing account, at say github,I was using my personal email, would I have to update my primary email to the one from paced?
Keeping a log like this is one kind of guard.
I've had to come up with my own set of tricks while working alone [0] but in general I think it helps to be a defensive developer.
[0] https://www.idiotlamborghini.com/articles/mastering_focus_as...