Tips on how to structure your home directory (2023)(unixdigest.com)
unixdigest.com
Tips on how to structure your home directory (2023)
https://unixdigest.com/tutorials/tips-on-how-to-structure-your-home-directory.html
171 comments
The worst offenders are CLI tools written by people on Macs that do not respect XDG, because it's not a thing over there. So every tool gets to pollute your dotfiles with its own stupid directory. .rustup, .mix, .npm, .yarn, etc.
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Weirdly, enough, golang is one of the only programming languages that actually has built-in support for a cross-OS config dir location: [os.UserConfigDir()][1].
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
function userConfigDir() {
switch (process.platform) {
case 'darwin':
return `${os.homedir()}/Library/Application Support`;
case 'win32':
if (process.env['APPDATA']) {
return process.env['APPDATA'];
} else {
throw new Error('%APPDATA% is not set correctly');
}
case 'aix':
case 'freebsd':
case 'openbsd':
case 'sunos':
case 'linux':
return process.env['XDG_CONFIG_HOME'] || `${os.homedir()}/.config`;
default:
throw new Error(`The platform ${process.platform} is currently unsupported.`);
}
}
> people on Macs
> pollute
gah, reminds me of my pet peeve.
Insert a drive, it gets .Trashes ._Trashes .DS_Store .fseventsd .Spotlight-V100 and other nonsense everywhere.
And if you are ever trying to recover a failing drive, do not mount it on a mac.
> pollute
gah, reminds me of my pet peeve.
Insert a drive, it gets .Trashes ._Trashes .DS_Store .fseventsd .Spotlight-V100 and other nonsense everywhere.
And if you are ever trying to recover a failing drive, do not mount it on a mac.
>But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
The point of XDG is to take the guesswork out of it by having applications follow your stated preferences.
"I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in."
1. The files created by the installer, which are handled by the package manager. I can consult my package manager for files created by a specific package: pacman -Ql package_name.
2. The files created after installation (user preferences, plugins). The program should follow the XDG specification.
I don't worry about where my distro package manager installs files.
Some install methods are outside of your package manager, but try to touch /usr or /opt or /var, like NVIDIA sh scripts and the likes.
Some install methods are outside of your package manager, but try to touch /usr or /opt or /var, like NVIDIA sh scripts and the likes.
And those should be avoided as much as possible because they're always problematic and break randomly. How could they not break, when the package manager doesn't even know they exist or what they depend on?
Well now we're going around in circles:
1. External install scripts that put stuff in system directories are sloppy and we don't like them
2. External install scripts that put stuff clearly-named non-hidden directory in your $HOME are better than the above
1. External install scripts that put stuff in system directories are sloppy and we don't like them
2. External install scripts that put stuff clearly-named non-hidden directory in your $HOME are better than the above
In theory, all applications following XDG would also solve the problem, as it has only 1 configuration directory. (Although I admit some apps use the data directory which brings it up to 2)
XDG isn't enforced, it's voluntary. So there will always be lots of applications not following it.
wutwutwat(2)
IMO, the GOPATH itself is a terrible design. It makes you mix together things from unrelated projects in the same directory structure, instead of each project having its own independent directory structure, like on every other programming language. The GOPATH design goes so much against the way I organize my projects, that it's the main reason I have never got interested into learning Go.
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
GOPATH actually made me realize that the
~/src/$host/$owner/$repo
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
~/src/$host/$owner/$repo
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
Can you expand on this a bit please? What does $host mean? Why would I need that for a purely local project that is only created for my own use?
And what about grouping projects? E.g. "personal", "work", etc. And where in the structure are languages? Is that the overarching directory?
> What does $host mean?
In Go parlance, it would be the remote host where the repository is hosted, e.g. github.com, dev.azure.com, golang.org, etc.
> Why would I need that for a purely local project that is only created for my own use?
If nobody else is using your purely local project, and you're sure nobody will ever use it until the end of time, then I guess you could just use "~/src/$HOSTNAME/$USERNAME/$PROJECTNAME". Otherwise, it would be wise to setup a remote repository ahead of time.
Go has a strong opinion that, in this day and age of distributed computing, projects should be online-first, so they can be easily used as dependencies. One of the nice consequences of this opinion is that Go dependencies can just be specified in the import statement - e.g. using grpc dependency is just:
> And what about grouping projects? E.g. "personal", "work", etc.
Assuming you only use one repository hosting service and have one username, all your personal projects would be under "~/src/$PERSONAL_HOSTING_SERVICE/$USERNAME/", and your work would be under "~/src/$WORK_HOSTING_SERVICE/$WORK_ENTITY/" or something like this.
> And where in the structure are languages?
It isn't. That's either a bug or a feature. If it's a bug, you could just do the whole thing by language, e.g. "~/src/go/", "~/src/java/", etc.
In Go parlance, it would be the remote host where the repository is hosted, e.g. github.com, dev.azure.com, golang.org, etc.
> Why would I need that for a purely local project that is only created for my own use?
If nobody else is using your purely local project, and you're sure nobody will ever use it until the end of time, then I guess you could just use "~/src/$HOSTNAME/$USERNAME/$PROJECTNAME". Otherwise, it would be wise to setup a remote repository ahead of time.
Go has a strong opinion that, in this day and age of distributed computing, projects should be online-first, so they can be easily used as dependencies. One of the nice consequences of this opinion is that Go dependencies can just be specified in the import statement - e.g. using grpc dependency is just:
import "google.golang.org/grpc"
> And what about grouping projects? E.g. "personal", "work", etc.
Assuming you only use one repository hosting service and have one username, all your personal projects would be under "~/src/$PERSONAL_HOSTING_SERVICE/$USERNAME/", and your work would be under "~/src/$WORK_HOSTING_SERVICE/$WORK_ENTITY/" or something like this.
> And where in the structure are languages?
It isn't. That's either a bug or a feature. If it's a bug, you could just do the whole thing by language, e.g. "~/src/go/", "~/src/java/", etc.
> No need for pom.xml, requirements.txt, cmake.txt, or any other kind of dependency configuration. It just works.
...until the project decides to switch to another hosting provider. Which has happened more than once in the past; it used to be common to host projects in Sourceforge, for a while Google Code was common, now many projects are on GitHub, and it won't surprise me at all when another forge becomes the popular one. Individually, projects might switch between being self-hosted (in their own domain name) and hosted on a shared forge (using the forge's domain name).
IMO, it's a bad design. It forces the project's repository's location to become the project's official "name", that is, it mixes up location and naming. It's better to have an indirection layer to map the project name to the project location, like most other languages do.
...until the project decides to switch to another hosting provider. Which has happened more than once in the past; it used to be common to host projects in Sourceforge, for a while Google Code was common, now many projects are on GitHub, and it won't surprise me at all when another forge becomes the popular one. Individually, projects might switch between being self-hosted (in their own domain name) and hosted on a shared forge (using the forge's domain name).
IMO, it's a bad design. It forces the project's repository's location to become the project's official "name", that is, it mixes up location and naming. It's better to have an indirection layer to map the project name to the project location, like most other languages do.
> ...until the project decides to switch to another hosting provider
Or decides their github username looks better with an upper case letter (https://github.com/sirupsen/logrus/issues/570). Or for people who use their real name as their github name, updating their username after marriage, divorce, gender transition or whatever.
Or decides their github username looks better with an upper case letter (https://github.com/sirupsen/logrus/issues/570). Or for people who use their real name as their github name, updating their username after marriage, divorce, gender transition or whatever.
Or they create a github organisation (which might be even worse because the old repo probably still exists but could be stale)
Go already supports an indirection layer, commonly known as vanity URLs. It works by making a request to a domain owned by the project and parsing a meta tag in the response that points to the actual repository location. Of course, the problem is that few projects bother to set this up.
Go also provides a replace directive in the go.mod file [0].
[0] https://go.dev/ref/mod#go-mod-file-replace
[0] https://go.dev/ref/mod#go-mod-file-replace
> It's better to have an indirection layer to map the project name to the project location, like most other languages do.
Who takes care of the indirection layer when the upstream decides to switch to another hosting provider?
Who takes care of the indirection layer when the upstream decides to switch to another hosting provider?
The upstream who manages their name::location mapping?
Please refrain from using the awkward question mark, it's condescending and rude.
What do you do if they fail to update their name::location mapping, and the language doesn't provide a way to do it yourself?
At least in Go, when that happens we can just add a `replace` statement in the go.mod file:
What do you do if they fail to update their name::location mapping, and the language doesn't provide a way to do it yourself?
At least in Go, when that happens we can just add a `replace` statement in the go.mod file:
replace example.com/foo/bar v2.35.0 => example.org/baz/bar/v2 masterWhy have owner in there? Isn't the clear from the filesystem metadata, which also has the benefits of shared ownership groups etc.?
The "owner" could be multiple directory levels depending the hosting service. Gitlab lets you have arbitrary sub levels. The owner of the files also isn't necessarily related to the owner of the repo on Github.
Is there then a hierarchy of owners?
This hasn't been the case since go modules exist, if I understand your issue correctly: https://go.dev/blog/using-go-modules. You can additionally vendor each project's dependencies
In a situation like this, when I have control over /etc/passwd and can create accounts as I please, then I might have a single account, let's say /home/amturing, then (as a subdirectory) another account /home/amturing/amtgo.
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
https://news.ycombinator.com/item?id=31846902
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
https://news.ycombinator.com/item?id=31846902
Well then you'll be relieved to know that that aspect of GOPATH was made optional in 2018 and hasn't been the default since 2021.
Pro tip: don't put your stuff in your $HOME directory. $HOME is for applications to pollute. Put your stuff literally anywhere else.
$HOME is where my terminal and file manager start. It's not reasonable to give up on this prime directory real estate.
That's only two applications to configure though.
The `~` path (and `cd` with no arguments, to get there) are also part of the prime real estate. It's as if these polluters think you'll enjoy accessing their stuff via `~/<tab>` which might or might not be true.
Polluters probably assume you'll never access anything at all except through dedicated apps, and bookmarks. In my case it's mostly true but it's still icky and ugly to see a crowded home dir.
[deleted]
Yeah, but that is still one application. You might come fr enough with CDPATH and configuring .inputrc to do something smart with ^m and the %-character (for the root of your document folder).
If it will be enough, I don't know. It might be enough for some, but it is still something which can be done without getting the maintainers of openssh to agree with the idea of a clean ~.
If it will be enough, I don't know. It might be enough for some, but it is still something which can be done without getting the maintainers of openssh to agree with the idea of a clean ~.
I can’t tell if this is sarcasm or not.
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
I follow the above advice. My personal stuff is in $HOME/$MY_FIRSTNAME.
I can organize this directory any way I want to, e.g. by project, time, purpose, whatever. It’s my stuff.
The stuff in $HOME is then more “the computer’s stuff.” (E.g., on MacOS, Photos, Desktop, Downloads - that’s where the computer dumps that stuff.) The other place is where I put things.
This turns out to work quite well. It allows me to be unconcerned when some application wants to put something in $HOME.
I can organize this directory any way I want to, e.g. by project, time, purpose, whatever. It’s my stuff.
The stuff in $HOME is then more “the computer’s stuff.” (E.g., on MacOS, Photos, Desktop, Downloads - that’s where the computer dumps that stuff.) The other place is where I put things.
This turns out to work quite well. It allows me to be unconcerned when some application wants to put something in $HOME.
Exactly, or simply "$HOME/a" for less typing.
I too follow this advice. I am the only user on my system. I would put all my stuff directly into /home if that were an option which was unlikely to have huge gotchas at some point.
I read the original comment as recommending against putting _anything_ in $HOME. Your solution is far more palatable.
To me its a sane choice on my own single-user machine. I used to create a top level `/data` directory to put my stuff into (before Android was widespread :) ). Nowdays, my data resides in `/data/main`. This scheme has served me well for more than 10a and through various OS migrations (Windows -> Ubuntu -> Debian).
I don't want to have all this `.cache` and similar stuff to take up my expensive backup storage, nor do I want to waste time cleaning up after all the applications that create/expect files in `$HOME` or think this a sensible place to put executable files. I don't want arbitrary applications (heard of tracker?) to start indexing my files and waste CPU on that etc.
Also I don't believe in /home being portable across multiple Linux distributions because e.g. different application versions might require config files to be migrated. Hence I opt for `/data` that can safely be shared across multiple systems because it contains actual data and not some OS-related state files.
On multi-user systems, the `/data` approach doesn't scale. There, I prefer to do just as boomboomsubban writes: I create a subdirectory in $HOME (usually, I call it `wd`, the "working directory") and treat that like my local `/data/main`. For large/temporary files (e.g. application downloads etc.) I add a `$HOME/large` directory to keep the `wd` reasonably small in such cases such that I can again include it in backups without having to worry about the size.
YMMV
I don't want to have all this `.cache` and similar stuff to take up my expensive backup storage, nor do I want to waste time cleaning up after all the applications that create/expect files in `$HOME` or think this a sensible place to put executable files. I don't want arbitrary applications (heard of tracker?) to start indexing my files and waste CPU on that etc.
Also I don't believe in /home being portable across multiple Linux distributions because e.g. different application versions might require config files to be migrated. Hence I opt for `/data` that can safely be shared across multiple systems because it contains actual data and not some OS-related state files.
On multi-user systems, the `/data` approach doesn't scale. There, I prefer to do just as boomboomsubban writes: I create a subdirectory in $HOME (usually, I call it `wd`, the "working directory") and treat that like my local `/data/main`. For large/temporary files (e.g. application downloads etc.) I add a `$HOME/large` directory to keep the `wd` reasonably small in such cases such that I can again include it in backups without having to worry about the size.
YMMV
It's not sarcasm. It's a pro tip which makes your life easier.
Also $HOME is a system directory, and it belongs to your operating system (.bashrc and such), also to your DE, also about every program and standard that I am aware of claims it for itself to use (freedesktop and such).
Also $HOME is a system directory, and it belongs to your operating system (.bashrc and such), also to your DE, also about every program and standard that I am aware of claims it for itself to use (freedesktop and such).
[deleted]
> In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
No. You got it backwards. If you put your stuff in a folder (not in the $HOME folder, but practically anywhere else), then your stuff will be in 1 place, there won't be any name collision with the system, it won't be polluted, and such.
Putting your stuff in $HOME is exactly what you want to avoid.
No. You got it backwards. If you put your stuff in a folder (not in the $HOME folder, but practically anywhere else), then your stuff will be in 1 place, there won't be any name collision with the system, it won't be polluted, and such.
Putting your stuff in $HOME is exactly what you want to avoid.
My ORACLE_HOME is in /home/oracle. It's just supposed to work like that (leaving all of the /u01/app references aside).
I have given up on the idea of a clean home dir a long time ago. I have "networked" directories (pCloud,Dropbox) in my home for everything I care for, and those are organized "perfectly".
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
Same thing, wish that from the start unix had some standardized "applications put everything they want here" home directory that's separate from the "here user chooses to put their own files" home directory
At the start, unix commands didn't need or use configuration files. The world really was that simple, for a while. Now we have even simple desktop apps, that I have not changed a single setting in, insisting to write a whole directory tree of whatever data in my home.
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
Or ending up with multi-gigabyte ML models somewhere in deeply nested hidden directories in your homedir
I wish more developers would seek out and embrace libraries for saving things in sensible locations. Python has one called “platformdirs” that takes the guesswork out of where to save things automatically for the user’s platform. I have no doubt all the other major environments have something similar.
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
Standards XKCD I guess.
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
Standards XKCD I guess.
To alleviate the issue with most applications dumping their files in your home directory, i found xdg-ninja[0] to be helpful.
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
[0] https://github.com/b3nj5m1n/xdg-ninja
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
[0] https://github.com/b3nj5m1n/xdg-ninja
Beyond organization, I want to concisely back up and port between machines.
The `.config` folder is a big headache when trying to be strategic about backups due to apps putting gigs of session data there.
"session data" is not "config", consarnit! An app's "config" shouldn't be gigs large. Grumble grumble, grouse grouse, and so on. :^)
The `.config` folder is a big headache when trying to be strategic about backups due to apps putting gigs of session data there.
"session data" is not "config", consarnit! An app's "config" shouldn't be gigs large. Grumble grumble, grouse grouse, and so on. :^)
And it should be possible for `.config` to be read-only, provided you do not ask a program to change a persistent setting.
You would love nixos - all configs are readonly.
So much this, why do apps use .config like a place to store app data? We have .local and .cache for that
Because programmers like to have something like DATA_DIR - a writable directory managed by the program.
Thinking about multiple storage locations is effort and most users don't care about this too much anyway
Thinking about multiple storage locations is effort and most users don't care about this too much anyway
This is so individual; his solution is not useful for me, and my solution won't be useful for anyone else.
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
The only thing that matters to me is portability when I switch systems, and aware of work vs personal environments
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
I still put my .vimrc and .bashrc/.zshrc there.
I think it's a nice idea, but don't like the media structure into family and so on. Looks like he's going to have a ton of problems with duplicates and then edited duplicates down the line, which can easily be mixed up, and then you lose your edits and so on.
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
Media-specific content-addressed stores (ideally as overlays which refer to but don't change your directory structure or file organization) seem to be the way to go with this.
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
Depends a bit on the use case. For personal photos, mobile photos and the like, I use Google Photo as well. But it just isn't suitable for professional storage or sorting IMHO, tho the face recognition is certainly neat. The best I've came across was FotoStation, but it's many years since I worked with those sorta things.
You may want to try TagSpaces https://www.tagspaces.org/ or TMSU https://tmsu.org/ which provide mechanisms for managing tags of arbitrary files (not only EXIF or ID3 ones).
Oh these ones are sweet! Thank you!
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
For me the way I use is as follows:
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.
I very much dislike capitalizing anything. I know it's only an extra key to hit, but it's annoying as all get out.
I personally would rather type "ThisIsSomeOtherProject" than "this-is-some-other-project" as advocated on the page. Approximately the same number of keypresses but fewer characters to read.
One major complaint I have with file systems is that too many directories start with the letter “D”.
Desktop, Dev, Downloads, Documents, Dropbox etc.
I thought about taking action about this, but as the author points about, many applications are quite opinionated on the matter.
Desktop, Dev, Downloads, Documents, Dropbox etc.
I thought about taking action about this, but as the author points about, many applications are quite opinionated on the matter.
I use `/src` to avoid this problem
I use “/Code”.
I once accidentally did an `rm -rf ~/` on a machine without backups and it removed everything alphabetically up until `~/Coding` (everything else being unimportant). So I fixed that by setting up backups and renaming it `~/Zoding` lol. Haven't been foolish enough to make that same mistake again, though.
Yay, I am quite safe, all my projects are nested inside "~/Work"!!
did that oops recently and it deleted `~/VirtualBox VMs/` while `~/Code` survived.
I actually have quite a simple structure that serves me well (your mileage might vary)
It is simple, works on any OS. Although on Linux I do have some helper scripts. And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
projects/
2023/
2024/
0000-something/
0312-other-project/
0419-hn-comment/
It is simple, works on any OS. Although on Linux I do have some helper scripts. And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
I do something similar for relevant folders, where I organize by time.
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
> 041924
I can accept YY instead of YYYY, but surely YYMMDD is preferable due to sort order.
I can accept YY instead of YYYY, but surely YYMMDD is preferable due to sort order.
The advantage of 041924 is that, when changing directories from the terminal, you can type 0419 and tab the rest of the way (99% of the time the only completion is 24). It's more efficient. If I prepended the year, I'd have to type out 240419 every time.
Why not use folder dates for that? It could be sorted.
projects/ | 01.01.2017
something/ | 01.01.2024
other-project/ | 01.01.2023
hn-comment/ | 01.01.2022My experience with metadata is that a lot of applications don't care about its integrity but will never mess with the filename unless that is the purpose of the application.
Demo a sync program and find that all the creation dates have changed to today!
Demo a sync program and find that all the creation dates have changed to today!
> My experience with metadata is that a lot of applications don't care about its integrity [...]
This has led me to formally maintaining significant metadata in the filename, for certain kinds of files; everything else gets lost or diverges. For instance, a book file might be named `Title [a=Jim Smith; isbn=9781234567890].pdf`.
This has led me to formally maintaining significant metadata in the filename, for certain kinds of files; everything else gets lost or diverges. For instance, a book file might be named `Title [a=Jim Smith; isbn=9781234567890].pdf`.
I feel like you're probably kidding, but I kind of like this.
Maybe not psyched about the shell-interpreted brackets, semicolons, and spaces. But the idea is interesting!
Maybe not psyched about the shell-interpreted brackets, semicolons, and spaces. But the idea is interesting!
No, I'm serious. There's no portable filesystem metadata, and while a few kinds of media files have facilities for internal metadata (e.g. EXIF and ID3), most don't. A file name is the only non-content that is easy to keep associated with any kind of file.
The syntax is definitely an imperfect compromise, but most of the things I use this for have titles that frequently contain spaces (books, papers, music, etc.) so quoting would be necessary anyway, and I made myself a Python package to help with escaping and do other utility tasks like normalizing ISBNs.
The syntax is definitely an imperfect compromise, but most of the things I use this for have titles that frequently contain spaces (books, papers, music, etc.) so quoting would be necessary anyway, and I made myself a Python package to help with escaping and do other utility tasks like normalizing ISBNs.
that's a good idea, especially if you don't want to keep a database to manage your library. In the piracy side of things we just make sure to name files something that can be used to look it up in the database and get metadata from that.
Although I have to warn you that ISBNs are not a perfect guid and they sometimes get reissued. Good enough if you aren't hoarding the world I guess.
Although I have to warn you that ISBNs are not a perfect guid and they sometimes get reissued. Good enough if you aren't hoarding the world I guess.
A lot of filesystems don't store file creation dates. Posix only requires last modification date (which for directories, is the last time a file/subdirectory was added, removed, or renamed), last access date, and last status change date.
because it requires some additional actions when navigating
Yeah, I've migrated to this + a 'process'-kind of folder structure. I'll have more duplicate data, but it saves my sanity
Re: backups
I had an experience where I attempted to install a new Mac from a Time Machine backup and my Mac couldn't see anything on the Time Machine backup. Called Apple support and found out about a rare bug where the install process can sometimes initialize the Time Machine backup instead of installing from the backup.
Luckily I had backblaze set up but it took a loooong time to get a few hundred gig restored.
Now I have Time Machine, BackBlaze and iCloud backups. Every now and then I dump everything to a .tgz on S3
I had an experience where I attempted to install a new Mac from a Time Machine backup and my Mac couldn't see anything on the Time Machine backup. Called Apple support and found out about a rare bug where the install process can sometimes initialize the Time Machine backup instead of installing from the backup.
Luckily I had backblaze set up but it took a loooong time to get a few hundred gig restored.
Now I have Time Machine, BackBlaze and iCloud backups. Every now and then I dump everything to a .tgz on S3
> Hyphen vs underscore in filenames and directory names?
> Originally, when I stopped using Windows back in about 1998, I was used to using spaces between words in filenames and directories. As I progressed into the world of Linux and BSD, I changed all spaces to underscores, but since I have done (and still do) a lot of web development I eventually settled on hyphens. I not only think it looks better, but the fact is that search engines interpret hyphens in file and directory names as spaces between words. Underscores are usually not recognized, and as such, their presence can negatively affect search engine optimization. Even though files in my home directory are my private files and not something I put out on the web, I have just settled on using hyphens everywhere.
I couldn't agree more. It's so practice to use hyphens when navigating on terminal that i think it should be a standard. It's even better than pressing <SHIFT-KEY> every time to select an underscored filename or <BACKSLASH-KEY> for spaces. Duh
> Originally, when I stopped using Windows back in about 1998, I was used to using spaces between words in filenames and directories. As I progressed into the world of Linux and BSD, I changed all spaces to underscores, but since I have done (and still do) a lot of web development I eventually settled on hyphens. I not only think it looks better, but the fact is that search engines interpret hyphens in file and directory names as spaces between words. Underscores are usually not recognized, and as such, their presence can negatively affect search engine optimization. Even though files in my home directory are my private files and not something I put out on the web, I have just settled on using hyphens everywhere.
I couldn't agree more. It's so practice to use hyphens when navigating on terminal that i think it should be a standard. It's even better than pressing <SHIFT-KEY> every time to select an underscored filename or <BACKSLASH-KEY> for spaces. Duh
That’s a minor thing I dig about Lisps: variables can be named foo-bar-baz, with nary a shift to type.
I have found it surprizingly difficult to categorize my files in a way that is elegant and intuitive. I've been meaning to look into FUSE for a tagging-based system instead of the current hierarchical one, although I suspect that the difficulty of categorizing things is unlikely to go away even with tags.
The underlying problem is that the way your brain works doesn't really map well into categories. Text searching sometimes works, but has the same problem.
Source: Nearly two decades ago we worked on this problem and read a bunch of research. We ended up making an app that let you input multiple facets to narrow things down, with the basic view being a calendar: https://nemo-docs.com/
We didn't have anyone marketing it, so it never really took off. The original plan was to build a serverless/P2P storage system underneath so sharing would be seamless and not require messing with servers, but never got that far.
Source: Nearly two decades ago we worked on this problem and read a bunch of research. We ended up making an app that let you input multiple facets to narrow things down, with the basic view being a calendar: https://nemo-docs.com/
We didn't have anyone marketing it, so it never really took off. The original plan was to build a serverless/P2P storage system underneath so sharing would be seamless and not require messing with servers, but never got that far.
> The underlying problem is that the way your brain works doesn't really map well into categories. Text searching sometimes works, but has the same problem.
What did you find that works ?
What did you find that works ?
Oh, I see. You put files/objects/data on a timeline. I wish the semantic desktop had brought us that. There's a firefox extension that conveniently displays your history in a calendar form and it's really useful.
Do you have any articles about that ?
Do you have any articles about that ?
I really like the idea behind this. That's what the Recent view should look like.
I always like the idea of tags, but whenever I get into a proper tagging system with thousands of entries I always end up having to regularly "refactor" my tags, which is fun exactly once and then it's just another goddamn chore.
The problem I have is that the optimal categorization seems to change depending on my current job and life situation. Tagging requires a whole new level of discipline ...
I am genuinely surprised tagging never got more traction in the native FS space. I use MacOS's tagging pretty extensively and Dropbox at least keeps them in sync, a lot of other services lose them.
I've tried using tags, but the effort isn't worth it for me. Between search, occasional housekeeping, and a decent folder structure I have everything I need without being coupled to Apple.
You can tag bookmarks too. No one does this, of course.
Tagging data is such a niche feature. But when it's useful, it's very useful.
Tagging data is such a niche feature. But when it's useful, it's very useful.
Having come up with a light structure and kept it for 7 years now, mine is:
media,
life,
edu,
data,
dev,
pix
The most heavily structured is /edu, everything else is a shitshow but I generally know where to find stuff. There are a few others that are dead. I don't bother putting dotfiles into this, but I probably should. I try to minimize directories with the same first letter for faster tab completion.
The most heavily structured is /edu, everything else is a shitshow but I generally know where to find stuff. There are a few others that are dead. I don't bother putting dotfiles into this, but I probably should. I try to minimize directories with the same first letter for faster tab completion.
What type of files do you keep in edu?
My Zotero database, thesis, and classes split up by semester with short names.
Anime porn
/life/money/taxes/tax17/receipts/
All the important business documents go in ~/horseporn of course.
In case anyone is looking for inspiration, my macOS homedir contains:
[1] https://github.com/ajeetdsouza/zoxide
[2] https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-...
# multiple organizations under Projects:
~/Projects/GitHub-org-name/repo-name
# temp files:
~/Scratch/0_jira
~/Scratch/1_temp
~/Scratch/2_builds
~/Scratch/9_archive
# Emacs org-mode files
~/Org
# Top-level screenshots
~/Screenshots
~/Projects/client_org/amazing-webapp
# Marky Mark and the Function Bunch
function gcd {
git clone $1 && cd "$(basename "$_" .git)"
}
function showpath {
echo $PATH | sed -e $'s/:/\\\n/g'
}
if [[ $OSTYPE =~ ^darwin.* ]] then
function brewup {
brew update && brew upgrade && brew cleanup
}
fi
function mc {
command mkdir $1 && cd $1
}
function mcd {
local TODAYDATE
TODAYDATE=$(date +%F)
command mkdir ${TODAYDATE} && cd ${TODAYDATE}
}
function mce {
command mkdir ${EPOCHSECONDS} && cd ${EPOCHSECONDS}
}
[1] https://github.com/ajeetdsouza/zoxide
[2] https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-...
The following directories always exist in my $HOME. Usually there are also a bunch of other work-in-progress files scattered around, waiting for me to either categorize them, delete them, or give up and move them to one of the catchall directories (archive, ongoing, tmp, or projects/scratch depending on what they are)
3dp -> 3d printing files
archive -> manuals, papers, website backups, etc
audio -> music and sound files
bin -> scripts and hand-installed binaries
build -> other peoples' projects that i'm building or hacking on
CameraUploads -> symlink to my NextCloud camera upload folder
Documents -> some random documents migrated from Dropbox days
Downloads -> Downloads, organized by year, with cur year in root
notes -> textfile notes
ongoing -> catch all to get rid of stuff that i am not quite done with yet
personal-records -> water & power bills, medical records, pay stubs, etc
pics -> photos (mostly screenshots) from this PC specifically
projects -> my own projects that i'm working on (mostly git repos)
tmp -> theoretically, safe to delete at any time .... theoretically
BACKUP.TODO -> a textfile showing what i still need to back up out of my digital empire
3dp -> 3d printing files
archive -> manuals, papers, website backups, etc
audio -> music and sound files
bin -> scripts and hand-installed binaries
build -> other peoples' projects that i'm building or hacking on
CameraUploads -> symlink to my NextCloud camera upload folder
Documents -> some random documents migrated from Dropbox days
Downloads -> Downloads, organized by year, with cur year in root
notes -> textfile notes
ongoing -> catch all to get rid of stuff that i am not quite done with yet
personal-records -> water & power bills, medical records, pay stubs, etc
pics -> photos (mostly screenshots) from this PC specifically
projects -> my own projects that i'm working on (mostly git repos)
tmp -> theoretically, safe to delete at any time .... theoretically
BACKUP.TODO -> a textfile showing what i still need to back up out of my digital empire
The “user-dirs.dir” setting is something I didn’t know was configurable. I thought I was always stuck with ~/Desktop, ~/Documents, …
Have slowly been using nixos as my primary driver (still in an emulated vm though). One thing I love here is the declarative nature. Lately have been experimenting with various desktop environments. I tried gnome and “it works” but have been fascinated with the idea of “ricing” my setup.
Currently using a Wayland compositor called hyprland to achieve something like this: https://github.com/outfoxxed/hy3?tab=readme-ov-file#demo
Community has experienced some drama though with fdo/x/wlroots. Always willing to make a change up if a better alt comes up though.
Have slowly been using nixos as my primary driver (still in an emulated vm though). One thing I love here is the declarative nature. Lately have been experimenting with various desktop environments. I tried gnome and “it works” but have been fascinated with the idea of “ricing” my setup.
Currently using a Wayland compositor called hyprland to achieve something like this: https://github.com/outfoxxed/hy3?tab=readme-ov-file#demo
Community has experienced some drama though with fdo/x/wlroots. Always willing to make a change up if a better alt comes up though.
You should know that "ricing" is a term that's based in the racist term "rice burner" which is used as a slur for Japanese cars imported into the US.
I bought the book "Building a Second Brain"[^1] by Tiago Forte and its recommended PARA methods is well-conceived, directly usable and tried and trusted for me. There is even a follow-up book from him about PARA only [^2]. I recommend to check it out (you'll also find many resources in the internet without the need to buy a book - although I highly recommend it and it would be an appreciation for the author)
[^1]: https://www.buildingasecondbrain.com/book
[^2]: https://www.buildingasecondbrain.com/para
[^1]: https://www.buildingasecondbrain.com/book
[^2]: https://www.buildingasecondbrain.com/para
I'm a big fan of PARA method, but I only use it in my note-taking app. I tried to apply it to filesystem, but it didn't work for me. So I went with his philosophy of keeping it simple and what works for you.
Instead of having a dotfiles git repository, I find it more useful to have my entire `.config` directory be in a git repository, and by default ignore everything. My .gitignore then looks like
I also make the .config directory visible; I use ~/config and symlink ~/.config to it.
/*
!.gitignore
!/dotfiles
!/i3
!/git
I also make the .config directory visible; I use ~/config and symlink ~/.config to it.
I prefer to either LUKS-encrypt the volumes used by ZFS, or use ecryptfs at the user level. ZFS ‘encryption’ has some nasty gotchas, such as not encrypting snapshot names (which just boggles the mind …).
I add ~/tmp (cleaned out on every boot), and use ~/.config/user-dirs.dirs to assign good lowercase names to everything.
And of course all code lives in ~/src.
I add ~/tmp (cleaned out on every boot), and use ~/.config/user-dirs.dirs to assign good lowercase names to everything.
And of course all code lives in ~/src.
Whenever I'm thinking about sorting and filesystems, I think of this post from Stephen Wolfram [0] and it's HN discussion [1]. The bit on filesystems is titled 'My Filesystem'. The basic premise is broad categories are better than narrow. It has constrained my urge to waste time hyper-categorizing things.
[0] https://writings.stephenwolfram.com/2019/02/seeking-the-prod... [1] https://news.ycombinator.com/item?id=26045380
[0] https://writings.stephenwolfram.com/2019/02/seeking-the-prod... [1] https://news.ycombinator.com/item?id=26045380
I use Johnny.Decimal [0] for my folder organisation needs. I'm not strict with it but it works for me.
[0] https://johnnydecimal.com/
[0] https://johnnydecimal.com/
I'm glad people keep writing these "how I set up my computer" articles because it's how I learned almost everything I know - thanks to the kindness of internet strangers.
anyway here's my idiosyncratic setup, would probably be terrible for anyone else:
everything goes in Dropbox (paid 2TB account)
my Dropbox subfolders are organized by broad filetype, e.g.
Re: concerns about "where apps store stuff" I mostly don't care because everything I care about is in Dropbox. If it's something I use a lot, its config is symlinked in wherever the app expects it to be - otherwise everything is a few `dired` commands away.
(On macOS `M-x locate` can be configured to use the builtin `mdfind` which works very well for finding things IME. I think it's what drives Spotlight.)
At some point I may set up 'recoll' and all that but so far I haven't needed it so I haven't paid the complexity tax of trying to configure that across Windows/macOS/Linux vs. good old locate/grep
FWIW because I limit myself to a "standard" set of apps (Emacs, browser, VLC, PDF/image viewer) the above setup works pretty much the same across Windows/macOS/Linux, so most of the time I don't have to care that much which system I'm typing into
anyway here's my idiosyncratic setup, would probably be terrible for anyone else:
everything goes in Dropbox (paid 2TB account)
my Dropbox subfolders are organized by broad filetype, e.g.
~/Dropbox/{Code,Documents,Videos,Downloads}
/home/rml/.emacs -> /home/rml/Dropbox/Code/personal/config/dot-emacs
/home/rml/.Xinitrc -> /home/rml/Dropbox/Code/personal/config/Xinitrc
/home/rml/Code -> /home/rml/Dropbox/Code
... etc
Re: concerns about "where apps store stuff" I mostly don't care because everything I care about is in Dropbox. If it's something I use a lot, its config is symlinked in wherever the app expects it to be - otherwise everything is a few `dired` commands away.
(On macOS `M-x locate` can be configured to use the builtin `mdfind` which works very well for finding things IME. I think it's what drives Spotlight.)
At some point I may set up 'recoll' and all that but so far I haven't needed it so I haven't paid the complexity tax of trying to configure that across Windows/macOS/Linux vs. good old locate/grep
FWIW because I limit myself to a "standard" set of apps (Emacs, browser, VLC, PDF/image viewer) the above setup works pretty much the same across Windows/macOS/Linux, so most of the time I don't have to care that much which system I'm typing into
It seems to me that directories are a fundamentally flawed system or organizing files. A natural way to start a directory structure is by basing it on the type of file (i e. games, music,
pictures etc) but that will quickly fall apart after you want your files of different "types" relating to X to be under the same directory. So now you can either create some symlink mess, include the wrong "type" of file in your directory tree, or restructure. The problem with restructuring is that it's a lot of work and the end result is not really worth it. What tends to happen (at least for me) is overly deep directory structures (and/or too much branching) that are tedious to navigate, and in the end have the same problem. Another option is to just bite the bullet and create a X subdir for each of your main directories that have relevant files for X which is what I do. But that really sucks and I wish there was a better alternative that actually worked with existing programs which assume a directory based organization. There are some alternatives that work on top of existing filesystems e.g. tag based systems but I haven't tried them yet and I suspect they have their own headaches.
I wish every OS had a space that is only and truly my space. No predefined directories, no config files that I did not put there myself, and no app should ever create files or directories there.
I settled a very long time ago to organize all my own data into projects under /proj and an alias `cdp` for `cd /proj`. That was what decades ago my colleagues were doing, and it has worked well for me ever since.
I appreciate the[1] home dir as a place for config files mostly and despise it for having become the system's junkyard otherwise.
[1] I'm hesitant to say 'my' because since everyone seems to put all their junk there, it does not feel like it is mine at all.
I settled a very long time ago to organize all my own data into projects under /proj and an alias `cdp` for `cd /proj`. That was what decades ago my colleagues were doing, and it has worked well for me ever since.
I appreciate the[1] home dir as a place for config files mostly and despise it for having become the system's junkyard otherwise.
[1] I'm hesitant to say 'my' because since everyone seems to put all their junk there, it does not feel like it is mine at all.
/data has been that area for me. It has been common practice since the late 90s to separate OS and data on completely different devices even
The homedirs belong on the OS drives and are managed by the OS. Anything you want to keep safe goes on the data volume where backups run.
It’s also very nice to just disconnect the data drives before any upgrades. You can’t corrupt a filesystem that isn’t even present to the system
The homedirs belong on the OS drives and are managed by the OS. Anything you want to keep safe goes on the data volume where backups run.
It’s also very nice to just disconnect the data drives before any upgrades. You can’t corrupt a filesystem that isn’t even present to the system
I would say that ~/Documents is pretty good here. The only trouble I have had is it being the default save location of some apps. But I haven't had something yet write there without any explicit action.
I am really envious on the FreeBSD' filestructure of OS'.
The simplicity is so obvious and I really like that you know what you have.
I manage dotfiles with Vorta backups. I don't want to deal with symlink management and such, and it's an easy way to say "These five manually selected dotfiles get saved". I have a separate Vorta profile just for them.
I have projects folders for each category of project, and each project gets a folder. Anything programming related, gets a git repo.
I used to have my top level folders by category, documents, code, etc, and that was a miserable idea. I much prefer sorting by project.
Any folder can have an "Archive" dir for old stuff.
I also have a Collected folder, for stuff that's I just found online and saved, sorted by category. Stuff like RasPi os images and sound fonts go there. It's like downloads but less ephemeral.
Then I have the one that people might not like... TheRuins. Whenever I do a clean reinstall of an OS, which used to be every few years, my old home dir would go there.
I don't want to just use a new home dir, because for one thing it's probably 95% stuff I don't actually need, and also it might have files for some old version of some app that might break the new one.
Stuff I actually DO want, I can manually move along into my new home dir out of the old Ruins.
The rest can just stay until I eventually need more space and move it into an SSD.
All of my media files live in a folder synced with SyncThing, as do a lot of other things.
Stuff I really want to keep gets replicated to my phone and tablet, as well as actually being backed up.
Finally everything gets backed up with Vorta. It's deduplication is amazing when you often move stuff around in a Ruins folder.
If I had the budget, I'd probably have a NAS with two disks, one as a backup target, and the other for general storage.
I have projects folders for each category of project, and each project gets a folder. Anything programming related, gets a git repo.
I used to have my top level folders by category, documents, code, etc, and that was a miserable idea. I much prefer sorting by project.
Any folder can have an "Archive" dir for old stuff.
I also have a Collected folder, for stuff that's I just found online and saved, sorted by category. Stuff like RasPi os images and sound fonts go there. It's like downloads but less ephemeral.
Then I have the one that people might not like... TheRuins. Whenever I do a clean reinstall of an OS, which used to be every few years, my old home dir would go there.
I don't want to just use a new home dir, because for one thing it's probably 95% stuff I don't actually need, and also it might have files for some old version of some app that might break the new one.
Stuff I actually DO want, I can manually move along into my new home dir out of the old Ruins.
The rest can just stay until I eventually need more space and move it into an SSD.
All of my media files live in a folder synced with SyncThing, as do a lot of other things.
Stuff I really want to keep gets replicated to my phone and tablet, as well as actually being backed up.
Finally everything gets backed up with Vorta. It's deduplication is amazing when you often move stuff around in a Ruins folder.
If I had the budget, I'd probably have a NAS with two disks, one as a backup target, and the other for general storage.
Most of both this article and the discussion is covering the simple case of home on a local hard drive, rather than the more interesting real-world situations with network-mounted home directories and multiple different HW/OS architectures you'd want to be able to have handled transparently. Which is fine.
For more... interesting network scenarios, it can be helpful to construct some variable, say, $HOSTABI, that includes tokens for both hardware and OS.
$ echo $HOSTABI x86_64-ubu-2204
Which can then be used inside architecture-influenced variables like ARCH, LIBRARY_PATH / LD_LIBRARY_PATH, PATH, as well as inside the user's own installation system (makefiles, etc) used for anything the user compiled. Great for just installing some new hw/sw combo, mount home, cd into "~/src" or equivalent and run "make install", and just get all your own code recompiled and installed for your new architecture.
This is a microcosm of what sysadmins on heterogeneous networks do, mostly for different hardware architectures, although this was a much more common situation in earlier years than more recently where Linux has taken over even more of the corporate/research world. I've found it pretty useful even at home, where I'm using my one home directory across my systems' different installations of Linux with different library requirements (although I should probably sell that SGI Onyx/RE - IRIX not Linux - that Wing Commander III was rendered on).
My home directory is usually populated almost exclusively with dotfiles and some threescore subdirs with 3ish-letter names like doc/ etc/ fun/ (I'm a gamer) git/ iso/ job/<site>/<recursehere> lib/ man/ pub/ (packages of my own programs) sbin/ src/ steam/ (yep) tmp/ var/ (notes, logs, etc) vr/ ... and so on.
It seems to work out alright. Contains about half a million subdirs and 4.2 million files. ;-)
For more... interesting network scenarios, it can be helpful to construct some variable, say, $HOSTABI, that includes tokens for both hardware and OS.
$ echo $HOSTABI x86_64-ubu-2204
Which can then be used inside architecture-influenced variables like ARCH, LIBRARY_PATH / LD_LIBRARY_PATH, PATH, as well as inside the user's own installation system (makefiles, etc) used for anything the user compiled. Great for just installing some new hw/sw combo, mount home, cd into "~/src" or equivalent and run "make install", and just get all your own code recompiled and installed for your new architecture.
This is a microcosm of what sysadmins on heterogeneous networks do, mostly for different hardware architectures, although this was a much more common situation in earlier years than more recently where Linux has taken over even more of the corporate/research world. I've found it pretty useful even at home, where I'm using my one home directory across my systems' different installations of Linux with different library requirements (although I should probably sell that SGI Onyx/RE - IRIX not Linux - that Wing Commander III was rendered on).
My home directory is usually populated almost exclusively with dotfiles and some threescore subdirs with 3ish-letter names like doc/ etc/ fun/ (I'm a gamer) git/ iso/ job/<site>/<recursehere> lib/ man/ pub/ (packages of my own programs) sbin/ src/ steam/ (yep) tmp/ var/ (notes, logs, etc) vr/ ... and so on.
It seems to work out alright. Contains about half a million subdirs and 4.2 million files. ;-)
I have two folders: "Development" where the git repos are, and "Downloads" where everything else.
I have had a very curated home taxonomy, than I give up. Now nearly all my digital life is managed in a graph of org-mode notes, org-attached. So I do not need to look for some file traversing a tree but a single org-roam-node-find typing something to reach a heading title. This plus a heavy note usage allow for a far better search, full-text in notes via ripgrep as well, working far better both for structured files, like regular bills, and casual files. A small bonus the two-level cache-like global attachment dir is essentially a nicely balanced tree, meaning if I decide (experimented than dropped since of no use for me) to index my files for instance with Recoll (Xapian comfy wrapper) the indexing is typically faster than in a classic curated taxonomy.
I remember I deleted a built-in directory like "Documents", and it lost its custom document icon, even when I created a "Documents" directory anew. No amount of googling / asking forums and IRC could bring my custom icon back. :)
(Linux Mint, Cinnamon)
(Linux Mint, Cinnamon)
https://man.archlinux.org/man/user-dirs.dirs.5.en
If you delete one folder the file $HOME/.config/user-dirs.dirs will be updated sooner or later, and point to $HOME/ and the icon will not be restored by recreating any folder.
Just change the values in there to whatever.
If you delete one folder the file $HOME/.config/user-dirs.dirs will be updated sooner or later, and point to $HOME/ and the icon will not be restored by recreating any folder.
Just change the values in there to whatever.
Whoa! I will try that, thank you!
calvin@bison:~/work$ tree -d | head
.
├── 2022
│ ├── 04
│ │ ├── 19
so i open my shell, it drops me into ~/t/, which is symlinked to ~/work/YYYY/DD/MM.
my GTK file picker is also setup to automatically give me the last 3 days on the bookmark bar.
Time is not the _best_ heuristic, but it's not bad.
screenshots from scrot all get dumped into ~/screenshots/ by date.
the only real 'folders' i keep around are ~/src/, ~/bin/ and ~/documents/, with the last being like, tax documents.
calvin@bison:~/work$ tree -d | head
.
├── 2022
│ ├── 04
│ │ ├── 19
Please confirm you are not a monster and that you meant /YYYY/MM/DD.
Now I'm wondering what month "19" is named...
the baha'i calendar has 19 months, so the 19th month would be Alá or Loftiness: https://en.wikipedia.org/wiki/Bah%C3%A1%CA%BC%C3%AD_calendar...
but then the year 2022 would be far in the future as the current baha'i year is 181.
that date in the baha'i calendar would be march 4th 3866 in the gregorian calendar
but then the year 2022 would be far in the future as the current baha'i year is 181.
that date in the baha'i calendar would be march 4th 3866 in the gregorian calendar
Well the 8th month ain't October
Super bummed I'm not getting the joke. What's the segue to 8th month (August) and October (10th month) from
calvin@bison:~/work$ tree -d | head
.
├── 2022
│ ├── 04
│ │ ├── 19
?octo is latin for 8.
just like septem is 7, novem is 9 and decem is 10.
so the parent comment is saying that while at one point it may have been possible to derive the name from the number of the month, that is no longer the case because names and numbers do not match up and therefor the name would not be undevigintiber. thought it might be septendecimber ;-)
just like septem is 7, novem is 9 and decem is 10.
so the parent comment is saying that while at one point it may have been possible to derive the name from the number of the month, that is no longer the case because names and numbers do not match up and therefor the name would not be undevigintiber. thought it might be septendecimber ;-)
I have something similar with ~/Stuff. ~/Stuff contains a folder per month, and ~/CurrentStuff links to the current folder. From there I have a simple shell-function `mkstuff` to create a folder ~/CurrentStuff/$DAY-"$1".
Naturally, it's a huge mess, but it helps if a colleague is like "Hey, didn't we have an issue back in August with the XYZ?". `ls ~/Stuff/2023-{07,08,09}` tends to find things again.
Besides that, I mostly have Projects/ (git projects I change) and Repos/ (git projects I look at) on my work laptop.
Naturally, it's a huge mess, but it helps if a colleague is like "Hey, didn't we have an issue back in August with the XYZ?". `ls ~/Stuff/2023-{07,08,09}` tends to find things again.
Besides that, I mostly have Projects/ (git projects I change) and Repos/ (git projects I look at) on my work laptop.
I'm surprised nobody has mentioned homesick yet. I prefer homeshick[0], a Bash clone, so I can avoid installing Ruby.
[0] https://github.com/andsens/homeshick
[0] https://github.com/andsens/homeshick
I'm shifting towards simply "not having" a home directory. Actual data is stored on other partitions, eg `~/git/project1` => `/git/project1`, `~/Pictures/Wallpapers` => `/data/Pictures/Wallpapers`, etc. The shell can be set up to spawn in some custom directory, $CDPATH can be altered for quicker navigation or completions, configurations are managed by Nix so I simply don't have a reason to look at my $HOME that often.
Programs can pollute it all they want, since I don't use it myself I also don't have to care about that.
Programs can pollute it all they want, since I don't use it myself I also don't have to care about that.
I used to worry about organizing files because searching was something I only did by hand. I still often do it by hand, but I know I am doing it wrong.
Anyway I don't worry about organization as much because a hand search might take five minutes. Organizing files is never done because nobody understands someone else's organization. When it comes to file organization, present self quickly becomes someone else and the Documents folder contains an oldDocuments folder contains a Documents folder with an oldDocuments folder and so on.
Date is the most reliable way to organize files. It is monotonic.
Anyway I don't worry about organization as much because a hand search might take five minutes. Organizing files is never done because nobody understands someone else's organization. When it comes to file organization, present self quickly becomes someone else and the Documents folder contains an oldDocuments folder contains a Documents folder with an oldDocuments folder and so on.
Date is the most reliable way to organize files. It is monotonic.
I hope someday we won't have to think about this anymore. I just want to write, code, and create without the overhead of organizing. I don't want to decide how to correctly "file" my cabinets, or undertake home dir refactors because my usage patterns changed.
I only have respect for the unixdigest author, but I can't imagine living in his shoes. My notes tend to have more than one "category" and I'm too indecisive to accept one folder structure for them. And my notes/categories morphs a lot too.
I only have respect for the unixdigest author, but I can't imagine living in his shoes. My notes tend to have more than one "category" and I'm too indecisive to accept one folder structure for them. And my notes/categories morphs a lot too.
> Actually, you SHOULD be paranoid about important data. I learned this the really hard way - a long time ago - when I had just finished writing a 200+ A4 pages book and then lost everything because of a silly mistake and had to start all over and write the book again!
>
> It is impossible to describe that very special feeling that arises when you have just realized what just happened! Noooooooooooooo! Pleeeeeeaseeeeee NOOOOOOOOOOO! (utter despair and disbelief - LOL).
I also have similar data structure for dotfiles... I've been using ~/bin, ~/dotfiles, ~/data, ~/enc (for encrypted files), etc. However, for projects and documents, I use Johnny Decimal system (https://johnnydecimal.com), which works perfectly for my use case. It helps me reduce the decision fatigue of deciding what to name the folder and filenames.
Reading the comments here makes me feel guilty. I'm sitting on probably a few hundred files and folders called something like tmp, tmp1, foo, foo23, foobar, testxyz, etc...
They all hold probably very irrelevant stuff and are safe to delete, and I have yet to resort to those files for rescue, but you never know! Every now and then I collect them and put them in an archive folder. I'm now at "archive10".
I've got my own little structure:
# projects I work on
code/
dotfiles/
# main obsidian.md repo that everything I learn dumps into
omega/
# public repos for local code spelunking, these remain untouched
repos/
The only things in my home directory (note: I use Windows, but the convention still applies) are stuff that demand to be in there, like savegame files and software like Discord which demand to reside in %appdata% or else.
My stuff are found literally everywhere but, usually in their own drives and partitions.
My stuff are found literally everywhere but, usually in their own drives and partitions.
Same. I’ve relinquished control over all the OS-provided directories to Adobe and Microsoft and go and whatever else assumes I want its crap in those locations.
Above someone jokes about just putting everything in Desktop, but honestly, it’s what realistically winds up happening. I can’t be bothered to manage files myself, really. Mostly, the apps I use know where their things are, I have a rudimentary system for archiving finished work, and the rest I can handle with search.
I’m just not in any way geared to be a file clerk or self librarian. Fortunately I have a machine that is very good at those things
Above someone jokes about just putting everything in Desktop, but honestly, it’s what realistically winds up happening. I can’t be bothered to manage files myself, really. Mostly, the apps I use know where their things are, I have a rudimentary system for archiving finished work, and the rest I can handle with search.
I’m just not in any way geared to be a file clerk or self librarian. Fortunately I have a machine that is very good at those things
Does it really matter when your home folder is a mess. I have thousands of files in my downloads folder. Hundreds of folders in my home folder.
I never run into issues. I occasionally use search to find old files or just sort by newest date.
I just use Spotlight search now, i can't remember the last time I really put any thought towards directory structure or anything like that (besides maybe my one "code" directory)
just put everything on the desktop, then its all in one place.
Exactly, when desktop gets too cluttered I store stuff in the Trash also.
Tip: refuse to use programs that don't use .config. refuse to use programs that don't use your home directory as a cache or tmp folder
These days I just put everything into ~/Documents and use fzf to find what I'm looking for.
Looks like there is some progress with $HOME/.mozilla to be more XDG base directory compliant.
> easily and quickly find what you're
Yes, that's why due to yet another totally unrelated quirk of history you
> have all the basic hidden stuff which ... such as .config
> but I only leave those dotfiles in place which work identically across the different systems I use
or you can use a better solution like chezmoi config manager and have templates to allow you to store changes even for those files while retaining the history of changes
But the most important piece of advice is missing: get an Everything-like utility (fsearch is a worse alternative on Linux) and bind it to an easy shortcut, that'd be very often the fastest way to find any file and not have to remember all those folder hierarchies (not that they aren't useful)
Yes, that's why due to yet another totally unrelated quirk of history you
> have all the basic hidden stuff which ... such as .config
> but I only leave those dotfiles in place which work identically across the different systems I use
or you can use a better solution like chezmoi config manager and have templates to allow you to store changes even for those files while retaining the history of changes
But the most important piece of advice is missing: get an Everything-like utility (fsearch is a worse alternative on Linux) and bind it to an easy shortcut, that'd be very often the fastest way to find any file and not have to remember all those folder hierarchies (not that they aren't useful)
For the last couple of years, I've only had these directories: dev, test and prod.
Whatever I am working on or learning goes in dev, whatever is ready for testing or experimentation goes in test. Everything else goes in prod.
But I have been thinking of expanding on this.
Thanks for sharing the article. I might gain some inspiration.
Whatever I am working on or learning goes in dev, whatever is ready for testing or experimentation goes in test. Everything else goes in prod.
But I have been thinking of expanding on this.
Thanks for sharing the article. I might gain some inspiration.
~/inbox/unordered/archive/chaos
The best way to organize your home is gnu stow!
Somebody needs to tell people to stop putting their websites in monospace.
[deleted]
good
The one that upsets me the most is the default directory for go modules `~/go`. This frustrates me so much. I refused to install any go apps or use it for development for years. I've unfortunately had to give in, and it can at least be overridden by setting `GOPATH`, but it is a terrible, terrible default.