fwiw: twice now, I've successfully gotten a pip package linked with CUDA 8 & built Tensorflow from source — once for Python 2 and another for Python 3. Both on an Ubuntu 14.04 system
Major supporter of this thinking, and did several sketches in this direction a few years ago: http://www.illucia.com/
Made of:
1) a suite of interconnectable videogames & apps and
2) physical patchbays to connect them
Came from thinking about software like modules in a modular synth. Playgrounds of interconnectable control structures. Was patching games into samplers, slaving text editors to drum machines, etc.