Manuel Bronstein wrote a book [1] about symbolic integration, but unfortunately it only covers the transcendental part. There is also a shorter "tutorial" from some workshop [2].
Unfortunately this is not complete, because it does not work with integrals that require algebraic extensions (they are more complicated than transcendental extensions).
"Boundary" refers to the boundary of Anti-de Sitter (AdS) space. The model that is typically studied is that of five-dimensional AdS space for which the boundary is four-dimensional. One can now formulate a quantum field theory on this boundary that is "dual" to the theory in the interior in the sense that there is precise dictionary between quantities living on the boundary and quantities in the interior ("the bulk"). "Holographically" refers to the difference in dimensions of the theory living on the boundary and the one living in the bulk.
[1] Manuel Bronstein: Symbolic Integration I, https://doi.org/10.1007/b138171
[2] https://www-sop.inria.fr/cafe/Manuel.Bronstein/publications/...