The creeping scourge of tooling config files in project root directories(github.com)
github.com
The creeping scourge of tooling config files in project root directories
https://github.com/nodejs/tooling/issues/79
213 comments
Yeah, I take a cluttered root dir over having to hunt down files I may not know exist in various places in order to fully understand a project I’m not the maintainer of.
Gets me every time when a maintainer puts Dockerfile in some random subdirectory.
If the root dir is full of dozens of files from all the tooling needed to build your software, maybe you have another problem than directory structure...
Gets me every time when a maintainer puts Dockerfile in some random subdirectory.
If the root dir is full of dozens of files from all the tooling needed to build your software, maybe you have another problem than directory structure...
The same argument is made for not adhering to the XDG spec for user configuration files. The result is you have no real sense of specific locations for config files, shared static data, and app-generated data. If an app follows the spec you know exactly where to look. Otherwise it’s anyone’s guess.
I’ll admit this is less of a problem in project dirs, since tooling usually confines itself to a single file. But tooling outputs often similarly pollute. For example, I wish Python dumped its artifacts in a single directory I could delete, instead of in every source subdir.
I’ll admit this is less of a problem in project dirs, since tooling usually confines itself to a single file. But tooling outputs often similarly pollute. For example, I wish Python dumped its artifacts in a single directory I could delete, instead of in every source subdir.
> The same argument is made for not adhering to the XDG spec for user configuration files
I think it's a fundamental difference in that the project repo should be self-contained and "owned", whereas arbitrary software I run really shouldn't be putting files in my home directory.
In a software repo, anything inside it belongs to the project.
I think it's a fundamental difference in that the project repo should be self-contained and "owned", whereas arbitrary software I run really shouldn't be putting files in my home directory.
In a software repo, anything inside it belongs to the project.
I think there are similarities -- the project's root is the project's, so why does eg the Travis CI system get to insist on having a config file sat in the project root?
Where should it go?
Just like the gitlab-ci.yml, it handles things related to the whole repo, it basically runs a script in the root of the repo anyway.
Sure, supporting other paths is useful (eg. in case of a monorepo), but this default seems sane. Though I'm interested in where others move it, and why.
Just like the gitlab-ci.yml, it handles things related to the whole repo, it basically runs a script in the root of the repo anyway.
Sure, supporting other paths is useful (eg. in case of a monorepo), but this default seems sane. Though I'm interested in where others move it, and why.
it's pretty common to have a .ci folder for your build scripts.
But nothing is fundamentally wrong either with XDG-esque directory structures.
.config/travis/
.ci/travis/
all the same to me, as long as it's consistent.
But nothing is fundamentally wrong either with XDG-esque directory structures.
.config/travis/
.ci/travis/
all the same to me, as long as it's consistent.
> the project's root is the project's, so why does eg the Travis CI system get to insist on having a config file sat in the project root?
Because it defines the project's CICI pipeline? Aka how the project is built, tested, delivered, and deployed?
I mean, that's a more fundament part of the project than which files should be automatically ignored when committing stuff to git (i.e., .gitignore)
Because it defines the project's CICI pipeline? Aka how the project is built, tested, delivered, and deployed?
I mean, that's a more fundament part of the project than which files should be automatically ignored when committing stuff to git (i.e., .gitignore)
With 3.8 python has PYthoncacheprefix where you can specify where the pyc files end up. Makes life so much better with docker
Do you know about the pyclean tool?
Since switching to git, I'm increasingly of the mind that clean) belongs to the source control (`git clean') rather than the build. After all, the source control knows what files belong to a bare checkout of the project and which have shown up since you got your grubby little paws on the code!
git clean has options to only clean ignored files, which generally does what you want. I could see there being weird situations where a file is .PRECIOUS (in make's terms) or where you have a file that's ignored but not a product of the build (a developer-specific config file?) where this doesn't work, but I haven't run into that yet, and it's freeing to not have to maintain a free target by hand.
git clean has options to only clean ignored files, which generally does what you want. I could see there being weird situations where a file is .PRECIOUS (in make's terms) or where you have a file that's ignored but not a product of the build (a developer-specific config file?) where this doesn't work, but I haven't run into that yet, and it's freeing to not have to maintain a free target by hand.
Exactly... the problem isn't the proliferation of config files, it's the use of so many tools that require those config files
So are you advising developers to use less tools, or maybe all-in-one toolsets? Presumably tools are used to support tasks that were otherwise more onerous and time-consuming and maybe neglected because of that.
>So are you advising developers to use less tools, or maybe all-in-one toolsets?
No, you're the only one arriving at that conclusion.
The whole point is that if you're going to adopt tooling then you'll need to configure them all, and that's fine with config files in the project root tree.
No, you're the only one arriving at that conclusion.
The whole point is that if you're going to adopt tooling then you'll need to configure them all, and that's fine with config files in the project root tree.
How about an all in one config file?
You have a standard JSON config file “.config.json” that resides in the root folder with a global object, with each key containing the config for each tool.
Collisions might be a concern, but you could use the tool’s url as the key so it’s unique. You are already seeing runtimes like Deno and Go do that for importing source code. Couldn’t the same thing apply for this config file as well?
You have a standard JSON config file “.config.json” that resides in the root folder with a global object, with each key containing the config for each tool.
Collisions might be a concern, but you could use the tool’s url as the key so it’s unique. You are already seeing runtimes like Deno and Go do that for importing source code. Couldn’t the same thing apply for this config file as well?
I'm pretty sure that I've read an aphorism along the lines that every configuration over time develops into an ad hoc programming language (similar to Greenspun's 10th rule so maybe I am just misremembering https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule)
At any rate the closest I could find to this saying was this HN post https://news.ycombinator.com/item?id=8092967 "Configuration files suck. Just use a programming language"
>but you could use the tool’s url as the key so it’s unique.
reminds me of XML namespaces.
on edit: formatting
At any rate the closest I could find to this saying was this HN post https://news.ycombinator.com/item?id=8092967 "Configuration files suck. Just use a programming language"
>but you could use the tool’s url as the key so it’s unique.
reminds me of XML namespaces.
on edit: formatting
That's how I configure my projects using dhall.
I'm working on a web application at the moment with a REST API, database, rabbitMQ, a few python workers, nginx, 2 react projects and a Gatsby project plus some docker composes for testing.
Everything is configured in the root file and then this is imported into smaller dhall files in the subprojects. In the build scripts I generate all the yaml and JSON from these, even the docker-composes and it's very ergonomic.
I'm working on a web application at the moment with a REST API, database, rabbitMQ, a few python workers, nginx, 2 react projects and a Gatsby project plus some docker composes for testing.
Everything is configured in the root file and then this is imported into smaller dhall files in the subprojects. In the build scripts I generate all the yaml and JSON from these, even the docker-composes and it's very ergonomic.
First I'm hearing of this tool and it seems really nice. Do you perhaps have an open repo that shows how you are using the dhall files? If I can reduce my config files to dhall + justfile I'd be quite happy.
> How about an all in one config file?
What an awful idea. I don't even know where to start enumerating the problems that this approach creates, and what for? I mean, just think about it. Have you ever had to comb through a config file to add support for something but while you were not sure what option dictated which and what impact this or that option had because documentation is ineffective or awful or even non-existent?
Now multiply that complexity by all the options added by all the tools expected to use a file, compounded by the fact that God knows how many tools also write to that file and may or may not stay in their lane when doing so.
Who in the hell wants to deal with that mess?
And what for? Just because someone wants to adopt a tool but has a problem with having to configure its own config file?
What an awful idea. I don't even know where to start enumerating the problems that this approach creates, and what for? I mean, just think about it. Have you ever had to comb through a config file to add support for something but while you were not sure what option dictated which and what impact this or that option had because documentation is ineffective or awful or even non-existent?
Now multiply that complexity by all the options added by all the tools expected to use a file, compounded by the fact that God knows how many tools also write to that file and may or may not stay in their lane when doing so.
Who in the hell wants to deal with that mess?
And what for? Just because someone wants to adopt a tool but has a problem with having to configure its own config file?
Not so sure if it "must" be _one_ config file.
I also don't think that the format matters more than it's use.
Reminds me of some good advice in The Pragmatic Programmer, should be (simple) text-files for editing and (regardless if one or multiple files), a single source of truth (e.g. if multiple files, not the same time the same data to edit in). This is roughly as I remember it and I found it always useful to apply.
If I see duplication, it's about fighting the rot and remove it.
If configuration files are generated, they don't belong under version control (there are some exclusions to this rule, e.g. pinning dependency versions for example when shared with the repository).
I also don't think that the format matters more than it's use.
Reminds me of some good advice in The Pragmatic Programmer, should be (simple) text-files for editing and (regardless if one or multiple files), a single source of truth (e.g. if multiple files, not the same time the same data to edit in). This is roughly as I remember it and I found it always useful to apply.
If I see duplication, it's about fighting the rot and remove it.
If configuration files are generated, they don't belong under version control (there are some exclusions to this rule, e.g. pinning dependency versions for example when shared with the repository).
Yes, less tools. Perhaps by making better use of existing tools
I’ve never had a problem using find and grep -R
Likewise here. I think that as far as defaults go, having these config files accessible in project root so makes it easy for people to see what tooling a project uses.
Suppose this pushes through and .config becomes more of a default than a conscious project choice. I don't see all tooling packages to follow-through immediately with it - causing a period where you still have the scourge of some config files in project root, while others are at .config.
Suppose this pushes through and .config becomes more of a default than a conscious project choice. I don't see all tooling packages to follow-through immediately with it - causing a period where you still have the scourge of some config files in project root, while others are at .config.
[deleted]
That's syntax war, the least useful aspect of programming with strong opinion from every party. And next time there would be "why every language has different config folder?".
I totally agree with this except - just when I was done with reading this thread I went back to the project I was on and I noticed there was some file in the root of the project red in VSCode so I go look at it, it's a JSConfig.json wtf is that I don't remember that config.
It says
{ "compilerOptions": { "baseUrl": "./" }, "exclude": [ "node_modules" ] }
I go google what it is, seems to be some VSCode thing added automatically maybe? https://code.visualstudio.com/docs/languages/jsconfig
but VSCode thinks it should be marked red, something is wrong with this config file I have no familiarity with. grrr.
anyway - I think at some point, especially on projects with lots of devs and lots of tooling, you look at the root and like art you know what you like or don't like and sometimes there will be a feeling that there are just too many files here for configuring stuff! But probably that indicates some other underlying problem, not a problem with having config files in itself.
It says
{ "compilerOptions": { "baseUrl": "./" }, "exclude": [ "node_modules" ] }
I go google what it is, seems to be some VSCode thing added automatically maybe? https://code.visualstudio.com/docs/languages/jsconfig
but VSCode thinks it should be marked red, something is wrong with this config file I have no familiarity with. grrr.
anyway - I think at some point, especially on projects with lots of devs and lots of tooling, you look at the root and like art you know what you like or don't like and sometimes there will be a feeling that there are just too many files here for configuring stuff! But probably that indicates some other underlying problem, not a problem with having config files in itself.
Yeah - I think that's a cue pointing to you that your understanding of the project at hand may be lacking in some regard. I usually listen to that signal and learn the heck out of the current tooling whose config I see on the root before proceeding with the subdirectories.
That feeling of mild anxiety in not knowing why these files are there can be useful, is all I'm saying.
That feeling of mild anxiety in not knowing why these files are there can be useful, is all I'm saying.
yeah or it's a cue that someone using VSCode created a JSConfig.json checked it in and it got pulled by the latest git pull.
> I don't get this, what's the problem here? Having config files in root means you know they can apply to all sub directories.
This makes sense for configs that could ostensibly do this, but that's only a fraction of them. Regardless of what we do about those (if anything), the other ones at least shouldn't be laid out like this.
This makes sense for configs that could ostensibly do this, but that's only a fraction of them. Regardless of what we do about those (if anything), the other ones at least shouldn't be laid out like this.
... “shouldn’t be laid out like this” doesn’t explain why it’s a _problem_ though. I’m curious about this as well. I think it’s fine as is.
Same reason why we have subdirectories in the first place and don't just shove everything into one directory. Same reason why you (hopefully) don't put everything on your desk and instead designate places for things, like in drawers. Same reason why we have a ~/.config folder now instead of just shoving everything into ~/ like we used to.
There is no One True Place, would you like drawers with my preferences and no way to override?
The list of configuration files/directories is known. Why can't those who bother present them as they desire? Instead of forcing their will on others.
There reason for ~/.config is separation from data and cache. And the reason behind XDG Base Directory Specification is a generic way to run application with another config `XDG_DATA_HOME=foo bar`.
The list of configuration files/directories is known. Why can't those who bother present them as they desire? Instead of forcing their will on others.
There reason for ~/.config is separation from data and cache. And the reason behind XDG Base Directory Specification is a generic way to run application with another config `XDG_DATA_HOME=foo bar`.
> Same reason why we have a ~/.config folder now instead of just shoving everything into ~/ like we used to.
Who/what is the "we" here? Last I checked my homefolder was still chock full of dotfiles in-use.
Who/what is the "we" here? Last I checked my homefolder was still chock full of dotfiles in-use.
Now, half of them are in ~/.config This is better because it means you have to look in more places.
Similarly, things also like to hide state in ~/.cache
If they didn’t then rm -rf .appname would restore it to default settings, and we can’t have that.
Similarly, things also like to hide state in ~/.cache
If they didn’t then rm -rf .appname would restore it to default settings, and we can’t have that.
I notice you chose to exclusively point out the negative aspects.
[deleted]
So copy them in there after you clone it?
This is the sort of issue that leads to “this page intentionally blank” type use of resources.
“When everything is a hammer...” when all you have to work with is a file system it always looks messy.
I’m more interested in making sure the right values exist in memory for runtime use.
Source, config, and comments could all live in one file and be specially delimited for look up too.
It’s all arbitrary at that point. This is really focusing on the wrong issues in computing.
This is the sort of issue that leads to “this page intentionally blank” type use of resources.
“When everything is a hammer...” when all you have to work with is a file system it always looks messy.
I’m more interested in making sure the right values exist in memory for runtime use.
Source, config, and comments could all live in one file and be specially delimited for look up too.
It’s all arbitrary at that point. This is really focusing on the wrong issues in computing.
I think it's the varying degrees of importance and surprise factor.
One dotfile has the maintainer's eccentric syntax highlighting preferences, another has critical build system settings needed to make any use of the code. Which ones do I really need to knand which are just conveniences for users of a particular IDE?
There are very few pieces of information where, to me, the natural place to look for it is "a dotfile at the root of the project." gitignore is an exception here.
One dotfile has the maintainer's eccentric syntax highlighting preferences, another has critical build system settings needed to make any use of the code. Which ones do I really need to knand which are just conveniences for users of a particular IDE?
There are very few pieces of information where, to me, the natural place to look for it is "a dotfile at the root of the project." gitignore is an exception here.
A voice of reason
I agree it wouldn't make sense to put the configs on a global folder like $HOME/.config but I don't see what would be the problem of putting them on a .config folder at the root of the project directory.
Recursive behavior would still work, the same way that Git can recognize that there is a ".git" directory in one of the ancestor directories.
Recursive behavior would still work, the same way that Git can recognize that there is a ".git" directory in one of the ancestor directories.
With .config folder at root of project then paths in configs are ../whatever/ instead of whatever/ and ../ is evil plus having ../ on front of every path is noisy.
Good point, never had a problem to place the configuration files where they belong to, be it in the project root or not, until I figured it out. That's what software configuration management is for, and SCM is obviously in place (given this is a reported issue on Github).
And if you allow me to build on the "Finally" note in your comment with a personal note:
Some _human readable_ content belongs into the root as well, e.g. a read-me, information under which conditions the code can be used (copying info) and similar, the important stuff. Otherwise it would mean it only needs to be machine read-able (which can be fine, too, especially for pure configuration management, still though a repository will most likely checked out by a human from time to time).
Uncle Bob raises this question from time to time: When you checkout a repository and look into the tree, what do you see?
The root is always an opportunity to show the projects' birds-view.
And if you allow me to build on the "Finally" note in your comment with a personal note:
Some _human readable_ content belongs into the root as well, e.g. a read-me, information under which conditions the code can be used (copying info) and similar, the important stuff. Otherwise it would mean it only needs to be machine read-able (which can be fine, too, especially for pure configuration management, still though a repository will most likely checked out by a human from time to time).
Uncle Bob raises this question from time to time: When you checkout a repository and look into the tree, what do you see?
The root is always an opportunity to show the projects' birds-view.
I disagree with regards to _human readable_ content.
We put "README"'s in the project root because it's configuration for most git repo viewer's e.g. GitHub.
If GitHub didn't have this auto nice formatting, I probably wouldn't add it, preferring a docs dir (for the general case).
Licenses is awkward, it's kinda config too really
Yes, that's why humans can read it on Github. Many people put even ASCII art into it (and restrain from actual HTML tags despite those are just fine in Markdown). So actually no disagreement from my side on that one, at least for me. Your mileage may vary.
Oh what a reminiscence to back then when exchanging software on floppy disks, and when the !readme.txt not written in your language, you found the "link" to the file in your "docs" folder often at the very top.
Or you just found the docs folder, in case no readme there.
Btw., this works on Github, too. The docs folder is normally directly visible above the fold while the read-me might not, especially when there is a long list of files which OP put on topic.
Oh what a reminiscence to back then when exchanging software on floppy disks, and when the !readme.txt not written in your language, you found the "link" to the file in your "docs" folder often at the very top.
Or you just found the docs folder, in case no readme there.
Btw., this works on Github, too. The docs folder is normally directly visible above the fold while the read-me might not, especially when there is a long list of files which OP put on topic.
The problem is someone anthropomorphized their code as “suffering” as justification to debate the color of the bike shed.
I 100% agree on this. But it's been pretty hilarious on how obstinate tool makers are about changing to Python's new pyproject.toml [1]
Python packaging is a bit of a mess already, so when I recently started a new small package I wanted to choose tooling that would not clutter, but a fair amount of tool makers were reluctant to allow code into their repo that would use the unified toml rather than a ton of separate file.
.config is probably a better solution than a single top level toml, IMHO, but far more important is doing something unified rather than continuing the pollution of the top-level namespace, which merely obscures the project structure.
[1] https://snarky.ca/what-the-heck-is-pyproject-toml/
Python packaging is a bit of a mess already, so when I recently started a new small package I wanted to choose tooling that would not clutter, but a fair amount of tool makers were reluctant to allow code into their repo that would use the unified toml rather than a ton of separate file.
.config is probably a better solution than a single top level toml, IMHO, but far more important is doing something unified rather than continuing the pollution of the top-level namespace, which merely obscures the project structure.
[1] https://snarky.ca/what-the-heck-is-pyproject-toml/
That link is a really good read. Better than TFA, tbh. And also particularly well written
> An interesting side-effect of PEP 518 trying to introduce a standard file that all projects should (eventually) have is that non-build development tools realized they now had a file where they could put their own configuration. I say this is interesting because originally PEP 518 disallowed this, but people chose to ignore this part of the PEP xD We eventually updated the PEP to allow for this use-case since it became obvious people liked the idea of centralizing configuration data in a single file.
Hilarious and indeed interesting
> An interesting side-effect of PEP 518 trying to introduce a standard file that all projects should (eventually) have is that non-build development tools realized they now had a file where they could put their own configuration. I say this is interesting because originally PEP 518 disallowed this, but people chose to ignore this part of the PEP xD We eventually updated the PEP to allow for this use-case since it became obvious people liked the idea of centralizing configuration data in a single file.
Hilarious and indeed interesting
This happens often in Node.js where you can put the configuration of some tools (e.g. babel, jest) inside package.json
I love pyproject.toml. I managed to get rid of setup.py, requirements.txt, requirements-dev.txt and MANIFEST.in, and I'm not far from having no setup.cfg soon.
All others stuff (CI/CD workflows, linters, formatters, code of conduct, funding, issue and PR automation, dependency bots, ...) moved to .github.
Now I have a clean and tidy Python project with a tiny 12-items root directory: https://github.com/kdeldycke/meta-package-manager
All others stuff (CI/CD workflows, linters, formatters, code of conduct, funding, issue and PR automation, dependency bots, ...) moved to .github.
Now I have a clean and tidy Python project with a tiny 12-items root directory: https://github.com/kdeldycke/meta-package-manager
Python is unified on a requirements.txt file with pip, to manage dependencies.
Not "a" requirements.txt file, sadly. I've seen packages with different requirements for building, installing, and testing. All with their own damned txt file. Oh! And don't forget the duplication in setup.py -- you'll want to put your requirements there, too.
As far as I know, Pip doesn't have the notion of dev dependencies, so you need at least two requirements[.dev].txt files.
> As far as I know, Python doesn't have the notion of dev dependencies
Python doesn't have the notion of dependencies.
Pip doesn't have dev dependencies, poetry does.
Python doesn't have the notion of dependencies.
Pip doesn't have dev dependencies, poetry does.
That's (the deps, not the dev deps) likely to change with pip 20.3! All hail the dependency resolver.
It's already available behind a feature flag.
https://pythoninsider.blogspot.com/2020/07/upgrade-pip-20-2-...
It's already available behind a feature flag.
https://pythoninsider.blogspot.com/2020/07/upgrade-pip-20-2-...
D’oh! Thanks. :)
[deleted]
I don’t think this is true. I’ve personally seen the following (and I’m positive there are others):
1. setup.py with self contained dependencies (this is my general preference)
2. setup.py loading from requirements.txt
3. Standalone requirements.txt
4. requirements.txt generated from requirements.in (or similar)
5. pyproject.toml + Poetry with a dependencies section
1. setup.py with self contained dependencies (this is my general preference)
2. setup.py loading from requirements.txt
3. Standalone requirements.txt
4. requirements.txt generated from requirements.in (or similar)
5. pyproject.toml + Poetry with a dependencies section
requirements.txt has a lot of deficiencies for package management (compare to the various features Yarn or Cargo offer). The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies, and even then there’s no verification that the content matches what is expected beyond the version number (other tools use a content checksum). This can cause a long tail of reproducibility and maintenance issues.
It’s also a tiny subset of what the parent describes.
It’s also a tiny subset of what the parent describes.
It also can't separate dev vs test vs prod dependencies, for example. Maintaining separate files for those is not exactly a convenient solution
I strongly recommend https://github.com/jazzband/pip-tools to solve this. It provides a simple script to take a requirements file and "compile" a full specification of all transitive dependencies. You check both files into the repo, point pip at the generated file, and manually modify the other one. It means you often don't need to pin requirements manually at all, and the versions will be explicitly updated whenever you choose to recompile your requirements.
When I first set up my requirements.txt's, I usually set up a venv, install the things that I need and then run "pip freeze" to get a list of all deps including transitive deps, and put them all in the requirements.txts.
I do sometimes feel that people are making rube goldberg machines out of their package management in an attempt to avoid just writing down all their deps.
I do sometimes feel that people are making rube goldberg machines out of their package management in an attempt to avoid just writing down all their deps.
>The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies
Isn't that what pip freeze is for?
Isn't that what pip freeze is for?
The trouble there is that if you freeze you can't unfreeze. A project needs the equivalent of both gemfile and gemfile.lock (unless you're using a system like maven that got things right the first time), but there's no standard way of having both of them in Python.
This is a big oversimplification. There are often multiple setup.txt files for different environments, virtualenv setup scripts, tox files, several different build config files, manifest files, and so on. Python probably has more magic config files than C#, which is quite an accomplishment.
But there are usually several versions of python with several pips, possibly in combination with anaconda with multiple environments, a combination of named and rolling releases, and a SAT solver that's always slow but occasionally spontaneously decides to become extra slow and extend your build time by hours.
Oh and I forgot about setup.py. That too.
Oh and I forgot about setup.py. That too.
I never have a unified requirements file.
I tried to downvote this, but I think it’s already downvoted to the max. There is so much wrong with this pattern.
Please use something like Poetry to manage deps.
Please use something like Poetry to manage deps.
The amount of pressure you feel telling you you are using too many tools should scale linearly with the number of tools you use. This discomfort you feel with seeing all of those config files is telling you something. Use fewer tools.
I tend to see a glut of tooling as a sign that a language has serious design flaws, as most important things should be done by the compiler, but I’d rather have JavaScript with proper tooling and types than nothing at all.
JavaScript isn’t the hero we wanted, but TypeScript is the hero we need.
JS is a great language for it's intended purposes but I still can't get onboard the TS train. Adding types post-hoc has never improved a codebase that I've seen but it has caused a lot more bad patterns to show up.
Simplicity is a virtue (at least sometimes).
Simplicity is a virtue (at least sometimes).
> JS is a great language for it's intended purposes
Which part of JS, as a programming language, do you see as great or even adequate when compared with contemporary programming languages?
> Adding types post-hoc has never improved a codebase that I've seen but it has caused a lot more bad patterns to show up.
That assertion makes no sense. I mean, all TypeScript does is impose arbitrary sanity checks on a programming language which is plagued for not having any. Where do you interpret the addition of sanity checks as a source of problems?
Which part of JS, as a programming language, do you see as great or even adequate when compared with contemporary programming languages?
> Adding types post-hoc has never improved a codebase that I've seen but it has caused a lot more bad patterns to show up.
That assertion makes no sense. I mean, all TypeScript does is impose arbitrary sanity checks on a programming language which is plagued for not having any. Where do you interpret the addition of sanity checks as a source of problems?
> Adding types post-hoc has never improved a codebase that I've seen but it has caused a lot more bad patterns to show up.
I mostly meant that I do all my greenfield JS dev using TypeScript. Out of curiosity, what bad patterns have you seen caused by adding types “post-hoc”?
I mostly meant that I do all my greenfield JS dev using TypeScript. Out of curiosity, what bad patterns have you seen caused by adding types “post-hoc”?
with ESLint and Prettier on top.
also don't forget:
- .eslintrc.json to configure ESLint
- .prettierignore because it tries to format package-lock.json
- prettier.config.json
- npm/yarn and package.json + node_modules
- .nvmrc to pin your node version
- storybook
- renovate to update dependencies
- .editorconfig to ignore node_modules
- mocha for testing and .mochaarc.js
- stylelint and .stylelintrc.json + .stylelintignore
And maybe throw in Jest, tsconfig.json, a gulpfile, and some other stuff.
- .eslintrc.json to configure ESLint
- .prettierignore because it tries to format package-lock.json
- prettier.config.json
- npm/yarn and package.json + node_modules
- .nvmrc to pin your node version
- storybook
- renovate to update dependencies
- .editorconfig to ignore node_modules
- mocha for testing and .mochaarc.js
- stylelint and .stylelintrc.json + .stylelintignore
And maybe throw in Jest, tsconfig.json, a gulpfile, and some other stuff.
Maybe it’s just because I was raised on this ecosystem, but in my opinion the sheer amount of incredible functionality you get from all this tooling (at the low price of $0!) is worth 10-15 set-and-forgot files that don’t even show up when you type ls.
While I totally agree that the functionality of most tools is really impressive, this doesn't mean that the current standard of dropping 5 new config files into root for a tool is the best way to handle this. There are ways to do better. It seems that the maintainers of those tools just don't care.
Your comment makes me extremely happy that I haven't needed to deal with anything written in Node. This would drive me absolutely bonkers.
Hell, I thought Flask's root directory was getting a bit fluttery with .gitignore, .venv, .flaskenv, __pycache__, and requirements.txt...
Hell, I thought Flask's root directory was getting a bit fluttery with .gitignore, .venv, .flaskenv, __pycache__, and requirements.txt...
My god, I love Elm.
Systemd pokes its head around the corner. Build tools? Its stomach rumbles. The beast hungers.
So my project will have var/run/.build-unit?
I think the problem isn't the existence of the config files (since they don't change often), but the fact that they take up a disproportionately prominent place in the root directory (since most configure a tool that handles one particular problem that not everyone will have strong opinions about).
Having lots of tools is, in some ways a good thing. That's how Unix was designed: have lots of little tools, each designed really well for its job. If you have a problem, and there is a tool for that problem, there's nothing wrong with using the tool to solve the problem. It may come with a learning curve, but what often happens without the tool is you end up evolving your own tool anyway, it's just not as well designed as the industry leader.
The problem described on this git issue is generally less of a problem on the Java projects I've worked on, since most of the equivalent configuration ends up in the pom.xml. The JS equivalent problem isn't a problem of too many tools in itself, but rather that no tool in JS has done what Maven has for Java.
Having lots of tools is, in some ways a good thing. That's how Unix was designed: have lots of little tools, each designed really well for its job. If you have a problem, and there is a tool for that problem, there's nothing wrong with using the tool to solve the problem. It may come with a learning curve, but what often happens without the tool is you end up evolving your own tool anyway, it's just not as well designed as the industry leader.
The problem described on this git issue is generally less of a problem on the Java projects I've worked on, since most of the equivalent configuration ends up in the pom.xml. The JS equivalent problem isn't a problem of too many tools in itself, but rather that no tool in JS has done what Maven has for Java.
Not sure I agree with this due to the fact that all projects are different and such their tooling requirements will be different. I don't think it's fair to just apply a blanket rule of 'use fewer tools' - this tends to end up with people writing their own implementation of existing tools which then turns in to tech debt so it's kind of a void argument.
I think this argument is more about utilizing minimalism in software design than always writing-it-yourself. IMHO many of us have seem to favor a form of feature maximalism, and perhaps we should be tempering our ambitions design-wise. Too much software has far too large a surface area.
For anyone doing typescript, there are just a number of tools you need to work with on a project. And then there's value added with things like eslint and prettier. Most of these configs are set and forget.
There's lots to complain about in JavaScript land but IMO having a bunch of config files in the root is a feature, not a bug.
There's lots to complain about in JavaScript land but IMO having a bunch of config files in the root is a feature, not a bug.
It's the same with libraries, IMO. Of course both tooling and libraries increase with the size of your project, but if either is disproportionately large compared with the size of the code involved, it's a bit smelly.
That's short-sighted. What about repos containing eg. a frontend and a backend? I did that with a Go/React project and I naturally ended up with: eslint conf, prettier conf, tsconfig, package.json, yarn.lock, editorconfig, Makefile, jest config, go.mod/go.sum, golangcilint.yml, Docker file/.dockerignore and travis.yml. Tell me, what should I discard?
And no, splitting the Go part and the React part is not an option as it makes releasing ten times harder.
Telling people to "just use fewer tools" might make you feel clever, but it's just snarky, really.
And no, splitting the Go part and the React part is not an option as it makes releasing ten times harder.
Telling people to "just use fewer tools" might make you feel clever, but it's just snarky, really.
Linearly? I’d say quadratically.
[deleted]
I'm not sure why this [1] pattern can't be adopted/extended. Put everything in a .config directory.
Search path for config is:
* /etc/config
* $XDG_CONFIG_HOME (~/.config)
* $PROJECT_CONFIG ($PROJECT_ROOT/.config)
* directories between $PROJECT_ROOT and $CWD for .config
* $CWD/.config
This allows for system wide, then user wide, then project wide, then directory hierarchy configs, with lower levels overriding higher ones.
Inside .config, each tool can have it's own chosen file, directory, extensions directory, so:
.config/$TOOL.{json,yaml,toml,ini,xyz}
.config/$TOOL/config.ext
.config/$TOOL.d/modular config files eg 00-module.ext
Most of the time for most projects, there will be system wide and user wide config, then a project wide config. Overrides lower in the hierarchy tend to be less used.
Yes, it means that there's a tree search for a tool to get its final config. Whether that's a real issue for performance or otherwise is a potential to remove some of the hierarchy/search.
[1] https://specifications.freedesktop.org/basedir-spec/basedir-...
Search path for config is:
* /etc/config
* $XDG_CONFIG_HOME (~/.config)
* $PROJECT_CONFIG ($PROJECT_ROOT/.config)
* directories between $PROJECT_ROOT and $CWD for .config
* $CWD/.config
This allows for system wide, then user wide, then project wide, then directory hierarchy configs, with lower levels overriding higher ones.
Inside .config, each tool can have it's own chosen file, directory, extensions directory, so:
.config/$TOOL.{json,yaml,toml,ini,xyz}
.config/$TOOL/config.ext
.config/$TOOL.d/modular config files eg 00-module.ext
Most of the time for most projects, there will be system wide and user wide config, then a project wide config. Overrides lower in the hierarchy tend to be less used.
Yes, it means that there's a tree search for a tool to get its final config. Whether that's a real issue for performance or otherwise is a potential to remove some of the hierarchy/search.
[1] https://specifications.freedesktop.org/basedir-spec/basedir-...
How conscientious.
And yup, conventions like this work well Linux and BSD's. To note ~/.config for many configs
These days you see this happening inside of .github/, where configs related to gh repos and actions go.
If tooling authors started universally recognizing .config/ as a directory where we could keep stuff, the root could be super clean.
How about just one file, across languages / CI tools / etc? a "project.toml" or "project.yaml" or "project.json"?
Legacy configs could be ported via YAML / JSON straight into the TOML sections. It'd work for package.json or tslint.json, but not stuff involving runtime, e.g. .eslintrc.js
And yup, conventions like this work well Linux and BSD's. To note ~/.config for many configs
These days you see this happening inside of .github/, where configs related to gh repos and actions go.
If tooling authors started universally recognizing .config/ as a directory where we could keep stuff, the root could be super clean.
How about just one file, across languages / CI tools / etc? a "project.toml" or "project.yaml" or "project.json"?
[tool.npm]
name = "frontend"
version = "1.0.2"
[tool.npm.dependencies]
react = "^16.3.1"
[tool.travis-ci]
# ...
[tool.eslint]
# ...
[tool.poetry]
name = "backend"
[tool.poetry.dependencies]
django = "~3.1.0"
Legacy configs could be ported via YAML / JSON straight into the TOML sections. It'd work for package.json or tslint.json, but not stuff involving runtime, e.g. .eslintrc.js
One directory, maybe .config, seems like a good idea. One file is asking for trouble. All it takes is someone writing their first command-line app mixing up > and >> and suddenly the user's config for every tool they use is gone.
It should be versioned, so it requires a little more effort to be destroyed
But some (most) config belongs in version control, and other parts are environment-specific: perhaps a different path to your database in prod vs development, or instructions for your IDE that may not be to the taste of someone else working on the project with the same IDE.
set -o noclobber
Windows goes a step further with AppData Local and Remote. Which is a good distinction for those of us who like to sync with roaming folder.
Sadly there's a confusing LocalLow and not all apps use AppData anyway. But it's something.
Sadly there's a confusing LocalLow and not all apps use AppData anyway. But it's something.
> AppData Local and Remote
I never understood the difference between both. What is it!?
I never understood the difference between both. What is it!?
I meant 'Roaming' sorry. And IIRC the roaming is for companies that use network storage or sync to keep people's settings no matter what workstation they use.
I like the idea of a single consolidated config file as long as:
1. It supports real comments (so not json) 2. It supports imports/includes so if it gets unwieldy it can be split up.
1. It supports real comments (so not json) 2. It supports imports/includes so if it gets unwieldy it can be split up.
Splittig it up so you can also determine the environment. I do this with Django settings the time: base/common, local, testing, production. The latter are very tiny and contain small bits of extra configuration.
Personally I like this approach way more than a bunch of if/else statements in a single file.
Personally I like this approach way more than a bunch of if/else statements in a single file.
Nah, it's far too common for tools to automatically update config files. A unified file would be asking for corruption, lost comments, trivial formatting and ordering changes, and more.
I think ultimately it comes down to personalities. I'm the kind of person that likes directories and folders. And having top level files feels clunky and disorganized. Like much organization, taken to the extreme it can be a form of procrastination to avoid doing real work. Yak shaving, if you will.
But opening the root project folder to just a list of subfolders gives me a very nice first impression.
Below is my list of config files for https://www.joyapp.com (Django + NodeJS). Not a hugely complex architecture, mind you.
Some notes: .editorconfig at least tries to consolidate this for some things. And pyproject.toml could help with Python. Still, definitely jarring and something I've noticed.
But opening the root project folder to just a list of subfolders gives me a very nice first impression.
Below is my list of config files for https://www.joyapp.com (Django + NodeJS). Not a hugely complex architecture, mind you.
Some notes: .editorconfig at least tries to consolidate this for some things. And pyproject.toml could help with Python. Still, definitely jarring and something I've noticed.
.coveragerc
.dockerignore
.editorconfig
.flake8
.gitattributes
.gitignore
.isort.cfg
.prettierrc
.shellcheckrc
jest.config.js
mypy.ini
package.json
pytest.ini
requirements.txt
shell.nix
tsconfig.json
tslint.json
webpack.config.ts
yarn.lockOn another note can we talk about the amount of projects that just dump their config/log files in the root directory as a 'hidden' file when you start them? Why? we have a .config for a reason.
Thankfully more and more projects are moving to respecting XDG_CONFIG_DIR:
The holdouts fall into the following categories:
1. Projects where the author doesn't care but would take a PR.
2. "My tool is super simple and I don't want to complicate it to read env vars or have platform specific config"
3. Projects which value cross platform consistency over consistency with the platform (tmux, though the next version does budge a little)
4. Projects which consider XDG as just "some desktop Linux thing" and don't consider themselves as caring about desktop that much or just that they predate XDG (SSH)
5. Projects who mix runtime and config and cached data in a single directory and don't want to split them out for proper XDG support.
The ArchWiki probably has the best centralised tracking of attempts to get XDG support implemented: https://wiki.archlinux.org/index.php/XDG_Base_Directory
The holdouts fall into the following categories:
1. Projects where the author doesn't care but would take a PR.
2. "My tool is super simple and I don't want to complicate it to read env vars or have platform specific config"
3. Projects which value cross platform consistency over consistency with the platform (tmux, though the next version does budge a little)
4. Projects which consider XDG as just "some desktop Linux thing" and don't consider themselves as caring about desktop that much or just that they predate XDG (SSH)
5. Projects who mix runtime and config and cached data in a single directory and don't want to split them out for proper XDG support.
The ArchWiki probably has the best centralised tracking of attempts to get XDG support implemented: https://wiki.archlinux.org/index.php/XDG_Base_Directory
6. Authors who haven't heard of XDG_CONFIG_DIR.
[deleted]
I think it’s so that all the configs float to the top and get out of the way. Agree that it’s annoying though.
[deleted]
Do you mean the system-level config folder? Won't that cause problems when A) you rename / move a folder or B) copy a folder to a different computer?
Nope, he means the "new" LSB preferred directory for user-level configurations. "New" being quoted here because it's around a decade old already.
Nit: ~/.config is specified by XDG, not by LSB.
But yeah, the most recent revision of the XDG basedir spec is dated November 2010; we've had it for a decade.
But yeah, the most recent revision of the XDG basedir spec is dated November 2010; we've had it for a decade.
Ehh, this is just moving the mess from one location to another. All these config files can be grouped into two or three conceptual things:
* Dependencies - Things required for your code to run, for example Dockerfile, package.json, yarn.lock
* Metrics - Measuring code quality, but the project works without these, such as jest.config.js, .eslintrc, .eslintignore
* Other - I don't recognize several config files here, and suspect some don't fall into the categories above, but am not sure. Also, README.md and .gitignore would fall into this category, but probably ought to remain at the top.
So instead of a generic "configs/" directory, if I was to organize these myself, I'd probably want at least two subdirectories so it would, y'know, actually be a bit more organized.
* Dependencies - Things required for your code to run, for example Dockerfile, package.json, yarn.lock
* Metrics - Measuring code quality, but the project works without these, such as jest.config.js, .eslintrc, .eslintignore
* Other - I don't recognize several config files here, and suspect some don't fall into the categories above, but am not sure. Also, README.md and .gitignore would fall into this category, but probably ought to remain at the top.
So instead of a generic "configs/" directory, if I was to organize these myself, I'd probably want at least two subdirectories so it would, y'know, actually be a bit more organized.
To me this is part of a larger problem: tools that expect a given directory layout, or even worse, that they are the sole tenant of a given repository. Just because my project contains some node.js code doesn't mean that I want to have a top-level, global node_modules directory.
Let me organize my repository files however I want and however best fits my project organization.
Let me organize my repository files however I want and however best fits my project organization.
No please.
Standardization is the key.
You (not you!) Don't know how to organize your project, and I don't want to know where are the config in each project I work on
this is the solution :) ... I miss rails where everyone knew where everything was supposed to go, a little like how much I love prettier now - no more stupid bike shedding conversations about coding style.
I don’t find it a scourge. I actually find that I can look quickly at a project and get an idea of the tools that they are using. This seems to be a bonus and really a few more files isn’t a big deal.
This is a UI problem, not a file location problem. GitHub and VS Code (both M$), could implement a nice way of grouping these files out of the of other files at the tip level.
Oof, I hate it when an IDE/editor tries to "helpfully" present the file tree differently than it actually is.
Like what the Windows file manager does. This is something that absolutely infuriates me every time I use Windows. It's actually difficult to figure out what directory you're actually looking at sometimes because ever the address input bar is lying to you.
Looks pointedly at Visual Studio.
Few pieces of development-related software frustrate me more than Visual studio.
The seeming disconnect between what's on disk, where on disk it is, and where and how it's displayed in the UI is cavernous. It also outright ignore things that are present in the folder unless you explicitly add it through the interface. God forbid you drop a file in, add it through the UI then move it because you discover you've dropped it into the wrong one of the 3 million nested folders large dotnet projects seem to generate.
Changing things on disk underneath it practically gives it an aneurysm. VSCode and IntelliJ IDE's handle the same situations without panicking, so why is VS so fragile?
Few pieces of development-related software frustrate me more than Visual studio.
The seeming disconnect between what's on disk, where on disk it is, and where and how it's displayed in the UI is cavernous. It also outright ignore things that are present in the folder unless you explicitly add it through the interface. God forbid you drop a file in, add it through the UI then move it because you discover you've dropped it into the wrong one of the 3 million nested folders large dotnet projects seem to generate.
Changing things on disk underneath it practically gives it an aneurysm. VSCode and IntelliJ IDE's handle the same situations without panicking, so why is VS so fragile?
> The seeming disconnect between what's on disk, where on disk it is, and where and how it's displayed in the UI is cavernous. It also outright ignore things that are present in the folder unless you explicitly add it through the interface. God forbid you drop a file in, add it through the UI then move it because you discover you've dropped it into the wrong one of the 3 million nested folders large dotnet projects seem to generate.
This is no longer the case for "new style" VS project files (used by default on .Net Core projects). It works sanely now.
This is no longer the case for "new style" VS project files (used by default on .Net Core projects). It works sanely now.
I think it's because of C++ in large part. For whatever reason there's been weird conventions in different code bases on how to structure headers vs source (like, put them in the same dir, or have an "includes" and "src" dir that mirror each other, or have an includes dir that doesnt match the src dir at all. Not defending it because it sucks, but I think .NET just inherited that weird interface because its what C++ did.
In my opinion that belongs to the problem category "the software
thinks it's smart and tries to simplify something for you".
Some programs try to "correct" your input, some hide or even remove settings so they can't confuse users, some produce all kinds of popups in order to help but only obstructing the view of some information or messing up UI focus in the process, and the list goes on.
This is the best way to make me use other software whenever I have the choice.
Some programs try to "correct" your input, some hide or even remove settings so they can't confuse users, some produce all kinds of popups in order to help but only obstructing the view of some information or messing up UI focus in the process, and the list goes on.
This is the best way to make me use other software whenever I have the choice.
This assumes everyone is using GitHub and VS Code - I think the main issue is applying it to the node ecosystem as a whole
I feel like a given tool deserves at best a single item in the root directory. If your tool needs more than one, you should be making a folder. I'd be totally on board with putting all those folders in another folder, but I'm iffy on the indeed the frustration of getting everyone to agree on one.
My head hurt thinking about how much efforts and coordination across multiple ecosystems required to pull it off. And if they do manage to pull it off, the reward is... slightly cleaner project top directory?
Fully agreed but if everything is in one folder out of the it doesn’t need to be hidden anymore, it could be “config” instead of “.config”. Maybe even “config” (project config inside version control) and “.user-config” (project config by developer not inside version control). The issue hints at this but deliberately descopes user configs. Files inside the folders could have a dot depending on if they are normally edited throughout development or normally left alone.
It's not a scourge. It's a sensible default location. Most tools let you explicitely specify a config file path if you want to store it elsewhere.
Also, please don't use dotfiles. There is no reason to hide such important files.
It's nothing to do with importance and everything to do with how often you need to see and access the file.
My .vimrc and .bashrc are super important, but I rarely need to edit them and I never need to be reminded of their existence, so having them show up every time I ran ls in home would be annoying.
My .vimrc and .bashrc are super important, but I rarely need to edit them and I never need to be reminded of their existence, so having them show up every time I ran ls in home would be annoying.
Both of these should ideally not be in the home directory at all but under XDG_CONFIG_HOME.
As far as I can tell, the only reason this is even a problem is because github doesn't hide dotfiles
The arguments for using hidden files generally boil down to “they’re not hidden for me” (okay, so why are we using dotfiles then?) or “I don’t like clutter” (this is why there are “directories”).
I do like something to sort overhead from core code, whether `_` prefix or something else.
On which systems are they hidden? I hadn't even considered this issue because no file browser I've used in the last ~10 years has hidden dotfiles. Maybe Windows Explorer does, but that's a setting I typically change right away, so I don't even remember.
`ls` is probably the most important place they're hidden, but the Mac OS Finder honors this convention too.
Wow, I've used `ls -la` for so long that I didn't even know the default `ls` behavior. Thanks for reminding me.
They are hidden by default to protect newbies.
GitHub users can unhide them.
I think Windows has them hidden by default as well (in explorer).
Since Win7.
But Windows has a lot of problems with hidden files, so a lot of people just set it to not hide any.
But Windows has a lot of problems with hidden files, so a lot of people just set it to not hide any.
> On which systems are they hidden?
On most systems. There is a long-standing convention in the Unix-derived world that files and directories with names which start with a dot (this also includes "." and "..") are hidden in directory listings. The main exception is Windows, which inherited its conventions from MS-DOS; the convention there is that files and directories with the "hidden" and/or "system" attributes set are hidden from directory listings by default.
On most systems. There is a long-standing convention in the Unix-derived world that files and directories with names which start with a dot (this also includes "." and "..") are hidden in directory listings. The main exception is Windows, which inherited its conventions from MS-DOS; the convention there is that files and directories with the "hidden" and/or "system" attributes set are hidden from directory listings by default.
*nix systems hide dotfiles by default. Some file browsers may show them. It's been a while since Linux was a daily driver for me, but it would surprise me if any of the standard/common ones showed dotfiles by default. On macOS they're hidden in Finder. I don't know about Windows, I think it shows them by default.
Neither Dolphin nor Nautilus (the file browsers in KDE and GNOME respectively) shows dotfiles by default.
I think you have to configure osx finder to not hide them. Android seems to use the . prefix to hide images in the image gallery.
MacOS and many Linux distributions.
I mean, for most tools it would be pretty easy to look at .config/ first and then at the project root - and print something about the new best practice.
The whole thing is a little uncomfortable though. There is a clear isomorphism between the contents of the structure of, say, a JSON file, and the contents of a directory of structured text files (and other directories). Heck, you could build a little tool with a slider that moves all the data between a single file, and a deep directory hierarchy with lots of tiny text files as the leaves.
Personally I feel like the best solution is to somehow "indelibly" combine the tool with its config within a project, de facto removing all config options from the developer. In the worst case, it could be something as silly as a wrapper script with config embedded in there. Of course this is a kind of fiction, but a useful one. It feels nice to get to a point where you don't have to think about tooling. (The last time for me was like 2000).
The whole thing is a little uncomfortable though. There is a clear isomorphism between the contents of the structure of, say, a JSON file, and the contents of a directory of structured text files (and other directories). Heck, you could build a little tool with a slider that moves all the data between a single file, and a deep directory hierarchy with lots of tiny text files as the leaves.
Personally I feel like the best solution is to somehow "indelibly" combine the tool with its config within a project, de facto removing all config options from the developer. In the worst case, it could be something as silly as a wrapper script with config embedded in there. Of course this is a kind of fiction, but a useful one. It feels nice to get to a point where you don't have to think about tooling. (The last time for me was like 2000).
> In the worst case, it could be something as silly as a wrapper script with config embedded in there.
That's not a terrible idea. For me, I really like tools such as 'nft' that take advantage of the shebang line to solve more or less this exact problem.
That's not a terrible idea. For me, I really like tools such as 'nft' that take advantage of the shebang line to solve more or less this exact problem.
This is a non-issue, thus no need for a solution.
If you have too many config files, then you should use less tools.
Besides, most of them are already hidden anyway (because of dotfiles).
If you have too many config files, then you should use less tools.
Besides, most of them are already hidden anyway (because of dotfiles).
Hidden files are never hidden to developers because they need to be accessed and changed so often.
This is fine. It's practical, but a little ugly. Same discussion that we had the other day with Tailwind CSS. It's fine, it's very practical, but a little ugly.
[deleted]
I think a good fix would be hiding dotfiles on repository view like on GitHub. "ls" hides dotfiles from you by default, so why shouldn't GitHub. You can always add a checkbox "Show hidden files" or instead of completely hiding them put them in an accordion (12 hidden files) that people can click to reveal them.
I'm not a node dev, but our projects at work keep growing with helper files. We have a .buildkite to control the build, we have a makefile, a Dockerfile, a docker-compose file, go.mod and go.sum. These core files are about half of what we had a few years ago, so that is nice. The makefile references a dozen helper scripts though that are also checked into the repo (helpers to run tests, build images, publish images, clean up, etc). Most of these are vastly similar between different repos/services we run. I can't think of something better, and there is a part of me that enjoys the lack of magic: just follow the code starting at ./buildkite which will point to each make operation which will point to each helper script. The other part of me feels like, "I just want to write some code and as long as my project is structured right, everything else should be shared tooling _outside_ my repo."
It's probably too late to change every tool.
If the main problem is that, when you go look at a project on github or gitlab, you see a long list of config files rather than the code you're looking for, a cleaner solution is for the git sites to show the listing for a /src directory instead of / on the project page.
If the main problem is that, when you go look at a project on github or gitlab, you see a long list of config files rather than the code you're looking for, a cleaner solution is for the git sites to show the listing for a /src directory instead of / on the project page.
Or hide/collapse dotfiles like the ls command does.
Putting them in .config is something to work toward in the long term, but I like this as an immediate fix. A possible UX could be a single entry at the bottom of the repo root listing with "+ 6 hidden files, click here to show", that then expands out to to reveal the full listing when clicked.
Edit to add— someone suggests this on the linked issue, and it is mentioned as a concern that auto-hiding the files could lead to them being used to conceal malicious code. Not sure how significant of a concern that really is, but it's an interesting angle, anyway.
Edit to add— someone suggests this on the linked issue, and it is mentioned as a concern that auto-hiding the files could lead to them being used to conceal malicious code. Not sure how significant of a concern that really is, but it's an interesting angle, anyway.
You can also hide malicious code by putting it in a subdirectory, so I don't agree with their concern.
[deleted]
The root is the right place for project-wide settings.
GitHub and other tools should do what ls does and hide dotfiles by default. That’s why they are dotfiles.
Other project-unrelated config/metadata should follow the .git example and use dotfile config files, preferably yaml or something similar (not json, as it does not support comments).
GitHub and other tools should do what ls does and hide dotfiles by default. That’s why they are dotfiles.
Other project-unrelated config/metadata should follow the .git example and use dotfile config files, preferably yaml or something similar (not json, as it does not support comments).
Another option would be for Github etc to sort dotfiles to the bottom of the listing.
The suggestion in the thread to "support defining this in package.json" (https://github.com/nodejs/tooling/issues/79#issuecomment-664...) reminded me of autoconf. It also has the issue of lots of auxiliary files in the project root (but worse since they're not hidden files), and the solution there is that you can add "AC_CONFIG_AUX_DIR(aux)" to the configure.ac to make it look for these auxiliary files in the "aux" subdirectory instead of the project root (for instance, "aux/config.guess" instead of "config.guess").
I like how Gradle works.
Basically, if you want to add a new "tool" to your project, you add it as a plugin in `build.gradle`:
Basically, if you want to add a new "tool" to your project, you add it as a plugin in `build.gradle`:
plugins {
id "com.github.spotbugs" version "4.5.0"
}
spotbugs {
visitors = [ 'FindSqlInjection', 'SwitchFallthrough' ]
// more config
}
I feel like the is just unecessary OCD behavior. We get enough of that in software already
The good old "conventions" vs "configuration" debate. Seems like Javascript looks more and more like Java with all these manifests.
Though XML solved the issue of needing multiple config files long ago, with namespaces.
Though XML solved the issue of needing multiple config files long ago, with namespaces.
Now we just need to finish moving from REST to pseudo-RPC and we'll have come full circle back to the 90s. Does this mean I can put bevels on my UI buttons again?
*gRPC ;)
javascript ecosystem is just slowly (and badly) reinventing all of the "enterprise" features that they claim to hate about java!
If the organization of files in your repo is your biggest problem, you've obviously got excellent code and no tech debt.
In which case you probably have enough free time to build a tool that just moves files around at runtime. Organize your repo however you want, then run your tool as a wrapper for any other command, and it'll move all files into the places other tools expect them. Now you've solved the problem, which was that humans like aesthetically pleasing things that serve no useful purpose.
In which case you probably have enough free time to build a tool that just moves files around at runtime. Organize your repo however you want, then run your tool as a wrapper for any other command, and it'll move all files into the places other tools expect them. Now you've solved the problem, which was that humans like aesthetically pleasing things that serve no useful purpose.
There’s a solution to this problem: https://github.com/sindresorhus/hide-files-on-github
Additionally, your OS probably already natively hides files that start with a dot, so this is just a UI problem.
Please don’t “solve” this issue by moving files to a sub-directory. If anything, only leave non-config files there, it’s the obvious simple solution that most projects follow anyway.
Additionally, your OS probably already natively hides files that start with a dot, so this is just a UI problem.
Please don’t “solve” this issue by moving files to a sub-directory. If anything, only leave non-config files there, it’s the obvious simple solution that most projects follow anyway.
This is a problem I've encountered on my projects. I just want to nestle them all in a `config/` folder. Alas, lots of the tools really dislike if you would like a configuration in a spot other than the root directory. Suddenly you get to enjoy debugging all the times some functionality wants to be relative to the config directory instead.
I love the idea of a standard config file, but... Cross-project standardization in the JS ecosystem? Unlikely.
I love the idea of a standard config file, but... Cross-project standardization in the JS ecosystem? Unlikely.
The Rails community attempted to do something like this with the introduction of Webpacker in Rails 5. They backtracked after a few months. Most likely after realizing how many thousands of npm packages make tightly coupled assumptions to the location of things like ‘package.json’ and the amount of patches it would require.
In the screenshot in the article, 5 out of 8 are related to using JavaScript. Anecdotally, it does seem like more of a problem with Docker + JavaScript + Git in the same project.
I know some tools allow you to specify the location of your config files, and it would be nice if all tools started to do that.
I know some tools allow you to specify the location of your config files, and it would be nice if all tools started to do that.
It looks like the issue was opened against the tooling project of nodejs, so perhaps the author is trying to start with JavaScript. It is incredible how many of these files you get in a basic Node or React app however.
Rather than a .config/ folder how bout a .tooling/ folder ... as it is for the tooling. Projects usually have a config dir for server / environment configs. Would be easy to confuse the .config/ of tooling with the config/ for the app itself
I would assume all these tools to support a —-config or —-file parameter to specify where my config is. Couldn’t that be specified in package.json directly? That way you can move your config wherever you want.
[deleted]
This is a problem mainly for people who use GUIs.
When you use a terminal you don't really care how many files are in a directory, unless they have names that are annoying to autocomplete.
When you use a terminal you don't really care how many files are in a directory, unless they have names that are annoying to autocomplete.
If that were the case, why would we bother separating out code into multiple directories at all?
The reason we bother with modules and directories is to help organize code and ease discovery. And whether we use ls or a GUI, we all start out the process of discovery with a directory listing.
The top-level directory, where these files are, is often where there's the most flexibility in terms of how to lay out things. All this clutter hides the project organization and makes it difficult to discover what's going on. It was one thing when it was just autotools, but the proliferation of tools in the last decade has brought it over a tipping point, IMHO.
The reason we bother with modules and directories is to help organize code and ease discovery. And whether we use ls or a GUI, we all start out the process of discovery with a directory listing.
The top-level directory, where these files are, is often where there's the most flexibility in terms of how to lay out things. All this clutter hides the project organization and makes it difficult to discover what's going on. It was one thing when it was just autotools, but the proliferation of tools in the last decade has brought it over a tipping point, IMHO.
>If that were the case
It is the case: They don't bother me when I'm using a CLI, but when I use Visual Studio Code for instance, all those files are annoying when I'm looking for something/scrolling past them.
It is the case: They don't bother me when I'm using a CLI, but when I use Visual Studio Code for instance, all those files are annoying when I'm looking for something/scrolling past them.
This is not a new problem. You should see my home folder. I hardly ever look at it, I moved into ~/Downloads since at least I can keep that tidy myself.
Interesting discussion.
If it's about "cleaning up" before committing, a single script that moves files back and forth from project root to a .config path might also be an option...
If it's about "cleaning up" before committing, a single script that moves files back and forth from project root to a .config path might also be an option...
I'm using .config when I find it but for the last 15+ years on new projects, regardless of language I'd been making an `etc` directory and putting config in there.
what's the difference between /tool.cfg and /tool/tool.cfg?
if your project is a hodgepodge of tooling the problem is the hodgepodge of tooling in your project, maybe you need to nest modules or to split libraries, instead of changing everyone else sane defaults because modern hip toolchains can't handle dependents and nested subprojects without stepping on each other toes or requiring the full lib in a single place
if your project is a hodgepodge of tooling the problem is the hodgepodge of tooling in your project, maybe you need to nest modules or to split libraries, instead of changing everyone else sane defaults because modern hip toolchains can't handle dependents and nested subprojects without stepping on each other toes or requiring the full lib in a single place
The great thing about all these config files is as a barometer for the popularity of various bad flavor-of-the-month config file formats reinventing key=value.
[deleted]
I find many developers have an aversion to directories containing many files, even if those files semantically “belong there“.
I think our tools drive this aversion quite a bit.
I think our tools drive this aversion quite a bit.
A nice convention in python projects is that you can use setup.cfg or pyproject.toml for many tools and they each just use their particular section.
Simple. Add a new config to the project root directory that defines the config locations. Done. /jk
I do like the .config directory naming suggestion though.
I do like the .config directory naming suggestion though.
[deleted]
[deleted]
This is not what unexpected!
To me the scourge is tool config files being in the repo at all.
Ok I’ll make an exception for .gitignore or something that enforces house style or formatting conventions to make things proper for check in.
But an editor config? That’s so user-specific and shouldn’t affect (or should I say “infect”) the source code.
To me the scourge is tool config files being in the repo at all.
Ok I’ll make an exception for .gitignore or something that enforces house style or formatting conventions to make things proper for check in.
But an editor config? That’s so user-specific and shouldn’t affect (or should I say “infect”) the source code.
The .editorconfig specifies things like tabs vs spaces and the indent size. Everybody needs to use the same settings to avoid messing up the formatting every time they edit a file. Therefore, .editorconfig gets checked in so that everybody gets the same settings.
[deleted]
I put my noise in an etc/ subdirectory.
How come /usr/etc/ didn't become popular?
What's the advantage over plain old /etc ? I am always annoyed by /bin, /usr/bin, /usr/share/bin, and others. Why can't we consolidate?
I always wondered why /usr was not abandoned in favor of root
> Why don’t you move all /usr contents to / and forget about /usr?
> Because this introduces a lot of new toplevel directories, which all have to be mount points then to be shared across other hosts.
> Ok, but what about a root filesystem on the network and mounting local filesystems only?
> Then you would share the toplevel directory hierarchy among all hosts. Hosts would need to mount /etc and /var for host-only versions.
So it is about network boot. Sad. There is nothing "usr" about /usr, it is system installed packages - /sys or /System. And /usr is for "/usr/dmr" which is /home. At least no more /usr/local.
> Why don’t you move all /usr contents to / and forget about /usr?
> Because this introduces a lot of new toplevel directories, which all have to be mount points then to be shared across other hosts.
> Ok, but what about a root filesystem on the network and mounting local filesystems only?
> Then you would share the toplevel directory hierarchy among all hosts. Hosts would need to mount /etc and /var for host-only versions.
So it is about network boot. Sad. There is nothing "usr" about /usr, it is system installed packages - /sys or /System. And /usr is for "/usr/dmr" which is /home. At least no more /usr/local.
I think /etc/ is for system wide config, but I take back my original comment. XDG apparently has a standard for this nobody follows.
This is for config files inside a repo.
anddyyyu(3)
I also personally think it sucks.
Put everything in a folder, and use a namespace, e.g.:
Put everything in a folder, and use a namespace, e.g.:
org.organization-name.project.jsonnamespaces was the thing i was looking for without knowing what it was. it was a very happy day in my world when i was introduced to them.
If you want to put (most of) your config files in a .config folder, then most apps should support that
If you want to view your config files in a different way, then actually you're trying to solve a different problem. Perhaps hide show toggling or automatic grouping (see macOS desktop stacks) with your fs viewer would be a better area to tackle.
Finally, just wanted to say, everything at root level is usually config or a folder, so if I want to understand what tooling a project uses, I know immediately where to go, code is always in a "src" or similar. It sounds like this is a feature, not a bug.