dd built-in progress introduced in coreutils 8.24(git.savannah.gnu.org)
git.savannah.gnu.org
dd built-in progress introduced in coreutils 8.24
http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/dd.c?id=af2a4ed22594badd2719c0123441d69b17bd8328
50 comments
On BSD, unlike Linux, many utilities including dd(1) will dump a progress report when receiving SIGINFO. This was traditionally bound to ^T in shells, either by default or using stty(1).
It's SIGUSR1 on Linux, however there is no signal bound to ^T in Linux, nor is it apparently possible to bind new ones.
`killall -USR1 dd` works like a charm, or for the impatient: watch killall -USR1 dd
`killall -USR1 dd` works like a charm, or for the impatient: watch killall -USR1 dd
This example highlights a very useful alternative function of `watch`: to simply run a command on regular intervals without caring about the output.
Yea, but its a pain to open another tab and run another command and switch back to this tab to see the progress.
Why they didn't give a progress bar in the first place beats me.
Why they didn't give a progress bar in the first place beats me.
progress(1) is a separate utility on BSD; one of my favorites. (Although it is also built in to BSD ftp.)
I regularly use progress with dd, tar and other utilities.
At least one BSD dd now has a "msgfmt" option that outputs parseable, printf-style format strings.
Truthfully, I prefer it when these basic utilities are static except for fixing bugs. Using the new features in scripts means the scripts might lose some portability.
I regularly use progress with dd, tar and other utilities.
At least one BSD dd now has a "msgfmt" option that outputs parseable, printf-style format strings.
Truthfully, I prefer it when these basic utilities are static except for fixing bugs. Using the new features in scripts means the scripts might lose some portability.
There's also the pv(1) utility on Linux and OS X Homebrew:
DESCRIPTION
pv shows the progress of data through a pipeline by giving infor-
mation such as time elapsed, percentage completed (with progress
bar), current throughput rate, total data transferred, and ETA.
To use it, insert it in a pipeline between two processes, with the
appropriate options. Its standard input will be passed through to
its standard output and progress will be shown on standard error.
I use this often on OS X :)
I find this style of formatting specifiers an eyesore:
fprintf (stderr,
_("%"PRIuMAX"+%"PRIuMAX" records in\n"
"%"PRIuMAX"+%"PRIuMAX" records out\n"),
r_full, r_partial, w_full, w_partial);I assume you mean the `%"PRIuMAX"` bits. It is an eyesore, but it is how to portably write the specifiers for types whose size my vary across architectures. That is, blame POSIX, not GNU coding style.
Yeah, it's ugly, but it's sadly the only way to correctly do cross platform prints where the type may or may not be 64 bit.
The `PRIuMAX` macro in particular is not necessary. It expands to a format usable for `uintmax_t`, but you can also use `"%ju"`. Both were introduced in ISO C99.
(Unless your `printf` implementation doesn't recognize `"%ju"`, and the `PRIuMAX` macro and friends are defined in some non-standard system-specific manner.)
(Unless your `printf` implementation doesn't recognize `"%ju"`, and the `PRIuMAX` macro and friends are defined in some non-standard system-specific manner.)
Related, the recently posted "How to write C in 2016":
https://news.ycombinator.com/item?id=10864176
https://news.ycombinator.com/item?id=10864176
How is this supposed to work? The underscore is a convenience macro for the gettext localization function. But gettext works by looking up its string argument in the localization database, and here the string varies from one platform to another, so how can you be sure that the localization will be found?
I guess the answer is that you have to redo the localization for each platform that has a different expansion of PRIuMAX. But that's horrible: the localization will be identical except for the format specifiers, which will just be copied from the English version of the string. Is there support for automating this in the translation toolchain?
I guess the answer is that you have to redo the localization for each platform that has a different expansion of PRIuMAX. But that's horrible: the localization will be identical except for the format specifiers, which will just be copied from the English version of the string. Is there support for automating this in the translation toolchain?
What is the behavior of gettext() if the message is not found?
(This question is the answer to your question.)
(This question is the answer to your question.)
If the message is not found, gettext returns the argument unchanged, which would be a bug (English text incorrectly appearing instead of localized text). So that's not it.
The answer to my question is that xgettext and gettext have special handling for PRIuMAX and the other formatting macros in inttypes.h. http://www.gnu.org/software/gettext/manual/gettext.html#Prep...
The answer to my question is that xgettext and gettext have special handling for PRIuMAX and the other formatting macros in inttypes.h. http://www.gnu.org/software/gettext/manual/gettext.html#Prep...
Example (from Reddit post):
# dd if=arch.iso of=/dev/sdb bs=4M status=progress
61432564 bytes (61 MB) copied, 3.024017 s, 20.3 MB/s
Couldn't you just send a SIGUSR1 signal to see the progress?
That's more awkward and outputs a separate line per interrupt. Also to do that programmatically (to provide progress to a program using dd underneath) is tricky to do robustly due to the default disposition of SIGUSR1.
SIGUSR1 is really only a hack provided on systems that don't support SIGINFO (which is easier to generate and whose default disposition is easier to handle robustly)
You could but letting dd take care of it instead of doing it yourself is much better IMHO.
What if you started dd without status=progress ?
Updating dd to continuously dump text to the screen by default would probably break a lot of existing scripts.
Just on SIGUSR1 would be good enough to get an idea where dd is at. And it would be on stderr, not stdout.
> it would be on stderr, not stdout
Oh, that's true.
Anyway, what behavior do you want to see exactly? You can still send the USR1 signal and get the update behavior just like before. Were you just pointing out that there are cases where you still need to use that feature, or something else?
Oh, that's true.
Anyway, what behavior do you want to see exactly? You can still send the USR1 signal and get the update behavior just like before. Were you just pointing out that there are cases where you still need to use that feature, or something else?
If you're on BSD, that'll kill dd.
dd was always a big special to me. It's the only command-line utility that I know that takes the simple approach to argument parsing. I don't know why everyone wants to prefix their arguments with --
As mentioned in the online manual at http://gnu.org/software/coreutils/dd that command line syntax is inspired by the DD (data definition) statement of OS/360 JCL
For one thing, nobody wants to type:
mv opt=force old=oldname.txt new=/path/to/new.txt
I've been using dcfldd for years now, just because of the progress bar.
ddrescue also shows progress by default. Not sure if it has a comparable feature-set as dcfldd, but for data rescue/forensics, it definitely does the trick, and seems to be much more actively maintained.
i've found that pv is useful for this purpose. however i've never been quite clear on what the best way to use bs with pv is. is it
dd if=/dev/sda bs=1M | pv > sda.file
dd if=/dev/sda bs=1M | pv | dd of=sda.file bs=1MIf all you are doing is the above, then you don't need dd at all. pv will act like dd (but with stdin/out) and uses a sensible buffer size you can override. Your first example becomes:
pv --buffer-size 1m < /dev/sda > sda.file
-B BYTES, --buffer-size BYTES
Use a transfer buffer size of BYTES bytes. A suffix of "k", "m", "g", or
"t" can be added to denote kilobytes (*1024), megabytes, and so on. The
default buffer size is the block size of the input file's filesystem
multiplied by 32 (512kb max), or 400kb if the block size cannot be
determined.
You can also drop the `<` in that example, pv takes a file to read from as an argument:
pv --buffer-size 1m /dev/sda > sda.file
> Will do the same thing, and give an accurate % complete progress bar.
I'm not sure that's true, last time I tried to do:
That said, looking at the main page, it appears that this functionality is only available for the output end:
Note that if the input size cannot be calculated, and the output is a block device, then the size of the block device will be used and pv will automatically stop at that size as if -S had been given. (http://www.ivarch.com/programs/quickref/pv.shtml)
I'm not sure that's true, last time I tried to do:
pv < /dev/zero > /dev/sda
That said, looking at the main page, it appears that this functionality is only available for the output end:
Note that if the input size cannot be calculated, and the output is a block device, then the size of the block device will be used and pv will automatically stop at that size as if -S had been given. (http://www.ivarch.com/programs/quickref/pv.shtml)
Ah, I see what you mean, I was thinking in terms of images/files.
As an arguement it gives % of the input (64GB usb drive, 8GB image):
As an arguement it gives % of the input (64GB usb drive, 8GB image):
> pv /mnt/str/Downloads/stick-8g.img > /dev/sdi
916MiB 0:00:35 [1.88MiB/s] [========> ] 11% ETA 0:04:17
See: cat -v Considered Harmful by Rob Pike
This is a pretty useful feature for something that regularly copies large files/disks you'd want to see the progress of, in fact I'd say it should have been default behavior for years.
I'd make some crude comment about cargo culting, but do you have an explicit complaint? dd itself is derided by the plan9 crew for many more reasons than a progress bar, and the cli arguments are obtuse because they're a joke.
[0]: http://www.catb.org/jargon/html/D/dd.html
I'd make some crude comment about cargo culting, but do you have an explicit complaint? dd itself is derided by the plan9 crew for many more reasons than a progress bar, and the cli arguments are obtuse because they're a joke.
[0]: http://www.catb.org/jargon/html/D/dd.html
This is how it looks...
[sc-lx-phys-05 /home/cvogel]
$ dd if=/dev/zero of=/dev/null status=progress
11086075904 bytes (11 GB) copied, 7.000001 s, 1.6 GB/sI get "No repositories found"...
There's also this incredible tool that spies on processes to get real time progress:
https://github.com/Xfennec/progress
https://github.com/Xfennec/progress
I get "no repositories found" for that link.