vorakl

2 years agodiscuss

Then, this explains why. BTW, this is an expected Shell behavior, specified in the POSIX standard:

  If a filename begins with a <period> ( '.' ), the <period> shall be explicitly
  matched by using a <period> as the first character of the pattern or immediately
  following a <slash> character. The leading <period> shall not be matched by:
   * The <asterisk> or <question-mark> special characters

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...

Interesting fact about the FreeBSD tar's origins:

  GNU tar was included as the standard system tar in
  FreeBSD beginning with FreeBSD 1.0.
  This is a complete re-implementation based
  on the libarchive(3) library. It was first released with
  FreeBSD 5.4 in May, 2005.

https://man.freebsd.org/cgi/man.cgi?query=tar

My personal journey with FreeBSD began with version 5.3 (the first stable release on the 5th branch) in November 2004. I was completely unaware of such a significant tar change, and apparently I didn't care at the time. However, the entire 5th branch has been so revolutionary compared to the 4th branch that this change is just a drop in the bucket ;)

vorakl

2 years agodiscuss

This is an interesting one. What was the version of tar? Was it gnu tar? I wonder, how did you create that archive?

I'm asking this because I was trying to reproduce the same situation and everything seems to be working fine:

  $ tar -C /tmp/root3 -cvf test.tar .
  ./
  ./.config/
  ./.config/test
  ./.local/
  ./var/
  ./var/db/
  ./var/db/xbps/

or even like this

  $ tar -cvf test2.tar root3/
  root3/
  root3/.config/
  root3/.config/test
  root3/.local/
  root3/var/
  root3/var/db/
  root3/var/db/xbps/

No special options were needed. But, if I do it this way, then, there are definitely missing all dot directories:

  $ tar -cvf test2.tar root3/*
  root3/usr/
  root3/usr/bin/
  root3/usr/bin/xbps-uunshare

But this problem is not a tar's problem. That's only because "*" mask doesn't match dot files:

  $ echo root3/*
  root3/usr root3/var

For that purpose, you need to clearly add a dot:

  $ echo root3/.*
  root3/.config root3/.local

Thus, the solution might be

  $ tar -cvf test2.tar root3/.* root3/*
  root3/.config/
  root3/.config/test
  root3/.local/
  root3/usr/
  root3/usr/bin/

But, I'd rather stick to "-C dir/" option instead of relying on "*" mask in this case.

vorakl

2 years agodiscuss

I don't recall ever seeing this approach with sudo. Is it really becoming popular? It used to be just bash, e.g. `curl ... | bash`. But now there is nothing stopping you from putting sudo in that bash script and expecting with a high probability that NOPASSWD: is also there ;)

vorakl

2 years agodiscuss

I asked about options for GNU tar because there is a bit of strange behavior.

To add absolute paths to an archive, there is "-P" option, and man says it works only for creating archives: "Don't strip leading slashes from filenames when creating archives".

To extract absolute paths from the archive, you need to add the "-C /" option, and although the tool says "tar: Strip leading `/' from member names", it will still extract it in the right place because the paths become relative and -C puts them in the root.

However, if you add "-P" during the extraction (which is not mentioned in man), the "strip leading slashes" information disappears.

So if this message bothers someone, "tar -C / -xPf file.tar" will cleanly extract absolute paths from the archive ;)

vorakl

2 years agodiscuss

--no-overwrite-dir (Preserve metadata of existing directories)

This is a really good point! It solves the main problem. I'll add it to the article.

Although that's what, unfortunately, is set by default (at least in GNU tar):

--overwrite-dir (Overwrite metadata of existing directories when extracting (default).)

vorakl

2 years agodiscuss

How do you create a tar archive that "contains absolute paths"?

vorakl

2 years agodiscuss

Yes, but someone has to do the dirty work ;)

vorakl

2 years agodiscuss

This is a good and safe practice. And I think most people do it this way after cleaning up the mess of a badly built tarball at least once :)

vorakl

2 years agodiscuss

I wonder what is the origin of this "/bin/sh" and why it's often assumed that sh is in /bin by default. The standard is pretty clear about that:

"Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell built-in."

"Furthermore, on systems that support executable scripts (the "#!" construct), it is recommended that applications using executable scripts install them using getconf PATH to determine the shell pathname and update the "#!" script appropriately as it is being installed (for example, with sed)."

Ref: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s...

So, apparently on systems, where sh resides in /bin, "getconf CONF" will return /bin . However, this is not always the case. For example, on my Fedora Linux it is /usr/bin

  $ getconf PATH
  /usr/bin
  $ ls -l /usr/bin/sh
  lrwxrwxrwx. 1 root root 4 Jan  2  2023 /usr/bin/sh -> bash
  $ ls -l /bin/sh
  lrwxrwxrwx. 1 root root 4 Jan  2  2023 /bin/sh -> bash
  $ ls -ld /bin
  lrwxrwxrwx. 1 root root 7 Aug  9  2022 /bin -> usr/bin

vorakl

2 years agodiscuss

Certainly true. I just referred to this definition

"2.4 Other Language-Related Specifications

POSIX.1-2017 is currently specified in terms of the shell command language and ISO C. Bindings to other programming languages are being developed."

I've added a few words and links to other "utilities".

vorakl

2 years agodiscuss

Interesting! I just wonder... How would you do a conditional branching in sed?

vorakl

2 years agodiscuss

It's a good point! I've addressed it.

I referred here to the following definition:

"2.4 Other Language-Related Specifications POSIX.1-2017 is currently specified in terms of the shell command language and ISO C. Bindings to other programming languages are being developed."

But, you are definitely right that some of the tools, defined as standard "utilities", such as AWK, also have their own language. Although, it is only defined in the "utilities" section, https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a... However, "sh" is also defined under "utilities", although it has an extensive language specification that is provided as a separate document.

vorakl

2 years agodiscuss

> the Zephyr RTOS which claimed to have some kind of posix compliance...

The work is still in progress. It's also important to note, that Zephyr is intended to run on resource-constrained systems. There is not much logic in implementing fork() for example, since technically, there is only one process. So it implements a subset of the interface. Here is the link to the talk from the EOSS 2024 where Chris Friedt shares the current state of POSIX support in Zephyr: https://www.youtube.com/watch?v=nLnmXxXuS6w

vorakl

2 years agodiscuss

Thanks for pointing this out! I have addressed this important detail.

Submissions

How to destroy your OS with tar

A few facts about POSIX

[untitled]

[untitled]

[untitled]

comments