jamiedamien

4 yıl öncediscuss

Oh didn't know about the improved type 42 font support in the new matplotlib! That's good to know and I should check it out.

And good point, the PGF works just as well (results should be identical), but since all the plot information has to be compiled by latex, it ends up ballooning the compilation time of the tex doc and the matplotlib PGF page suggests that you can run into memory issues as well. I was doing this for a thesis with 50+ plots and so still wanted compilation to be fast.

I've suggested this as an improvement to matplotlib, but unlikely to be merged since maybe it's a bit hacky (although it's very similar to what Inkscape's export to LaTeX option does): https://github.com/matplotlib/matplotlib/issues/22297 (the backend file can be found here: https://github.com/matplotlib/matplotlib/files/7921801/backe...)

And the gs script is below:

  #!/bin/bash
  set -e
  set -o pipefail
  if [ -z $1 ]; then
    echo "Supply input output"
    exit 1;
  fi

  if [ -z $2 ]; then
    outfile="$(basename ${1} .pdf)-small.pdf"
    if [ -f $outfile ]; then
      echo "WARNING ${outfile} already exists."
      echo "Supply input output"
      exit 1;
    fi
  else
    outfile="${2}"
  fi

  gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH -dQUIET -dDetectDuplicateImages=true -r150 -sOutputFile="${outfile}" "${1}"

  pre_b=$(wc -c "${1}" | cut -d' ' -f1)
  post_b=$(wc -c "${outfile}" | cut -d' ' -f1)
  if (( $pre_b <= $post_b )); then
    echo "Original is smaller ($pre_b -> $post_b). copying..."
    cp "${1}" "${outfile}"
  fi

jamiedamien

4 yıl öncediscuss

Oh man, I've been down the rabbit hole of reducing matplotlib PDF sizes too many times. Ghostscript is great most of the time, but as mkl points out, it can make some PDFs bigger.

In particular, matplotlib plots that use points (markers) blow up in size quite a bit after processing through ghostscript, due to the way matplotlib re-uses spline information to draw the e.g. circles, where as ghostscript seemingly cannot / chooses not to (?). I recall something to do with xobject re-use...

I've also found that if you use type 42 fonts (helpful if submitting to a conference where the submission system doesn't accept type 3 fonts), matplotlib will not subset the font, resulting in increased file sizes.

So I use a similar ghostscript script, but one that also checks if the resulting file is actually smaller. If it's bigger, it just uses the original PDF. For files with lots of points, I've found that rasterizing just the points artist is a good solution (everything else in the plot is still vector), which allows for ghostscript to subset the type 42 fonts without the file-size explosion due to the points. Still, I wish there was a good way or script to e.g. just subset fonts in a PDF file, as well as processing a PDF to remove redundant fonts.

When including many PDF plots into a large LaTeX document, each PDF still comes with embedded fonts, which can increase the file size of the final PDF. Most of the fonts end up being duplicates. For this, I use a custom matplotlib backend that creates a PDF file with no text, together with a PGF file that specifies the position of each text. LaTeX then handles all the text rendering (which results in nice looking figures!), so each font is only included once in the final PDF.

jamiedamien

5 yıl öncediscuss

I suspect what happened is that the position and velocity estimates from the VIO system (cameras) were wrong (and perhaps wildly jumping) and since position and velocity errors are inputs to the attitude controller, the tilt oscillated as well. It's not that the IMU did a poor job estimating attitude (tilt), but rather that the attitude controller was asked to tilt in erratic directions to compensate for the erratic and incorrect position and velocity readings.

It's quite hard to detect when position and velocities are "obviously incorrect", especially when they come from VIO, where the optimization result can jump around in non ideal conditions, so I'm not surprised there was not a more graceful anomaly detection.

jamiedamien

6 yıl öncediscuss

I followed this during winter break my first year of college and loved it! Helped me learned Linux and co. pretty quickly and I used the machine as my main computer for the next 3.5 years of college.

Once you get X and google chrome it becomes pretty self sustainable. Updating the kernel is easy but upgrading the compiler / C library is a nightmare.

A lot of programs you might want to use provide binaries that makes things a bit easier, but there were some annoying days of dependency chasing if you have to install from source. I ended up writing some custom scripts to ./configure, make, and make install into a deb from .tar.gz.

About the initial LFS process, I had messed up a few steps pretty bad, and had to restart / go back one or two times. The worst mistake was embarking on the journey first by installing everything into a USB flash drive. Eventually data corruption / loss ruined the yacc binary (or similar? hard to remember) and I spent days trying to figure out what went wrong. After moving to an SSD it worked better :)

Definitely one of my favorite and most rewarding projects I've done.

jamiedamien

6 yıl öncediscuss

How is this similar / different to Johnson-Lindenstrauss? (https://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_...)

The principle seems very similar: find a simpler representation that preserves distance.

comments