B1gred · HackerTrans

I am a statistician working in the data sciences. I see lots of programmers show interest but do not have the depth of knowledge in mathematics and statistics. They can apply libraries and do 80% fine but do not have the educational background to side step assumptions and pitfalls.

B1gred

11 anni fadiscuss

It can mean anything, and that is why it is hard. Many old school statisticians can not work with anything other than csv, excel spreadsheets or basic sql queries. Munging is the conversion to a nice format that can then be used for analysis.

B1gred

11 anni fadiscuss

It is slow. And it is ok. Very few times will R ever beat any other language. Usually it is not off by much, but especially if coded by a novice using for loops vs apply functions can make is 100 -1000 x slower.

Another example is the immutable structure that causes R to be a memory hog. Creating copies of data everywhere. But, again if you plan well and execute the 'best' solutions you can avoid the giant pitfalls but will rarely ever beat a equally well written python equivalent.

B1gred

11 anni fadiscuss

Depends on coworkers. When in school, my professors couldn't do anything if it wasn't in a nice csv file. But, us data scientist / statistical programmers are well versed in digesting data in almost any form, especially a database. When on a new project, I just get handed new ip addresses and login information and I am off.

B1gred

11 anni fadiscuss

I program in R 80% of my day. I have experiences in all the major alternatives but keep returning to R. It has one huge flaw, being slow but otherwise is fantastic to work with and has a vibrant community.

The bigger issue is that while R is liked by statisticians it lacks many of the features for the software development. We run across difficulties with logging, version control of packages, speed, size of docker image, build time etc. But, with these drawbacks I keep coming back because I develop faster and better in R.

B1gred

11 anni fadiscuss

Python has a steeper learning curve and is not as curtailed to simple data analysis. Many use Rstudio (an ide) and use the import data, and other tools to make then skill entry even lower.

Also, mathematicians and statisticians think functionally and the general attitude in python is to do object oriented programming while R is strictly functional programming with a little bit of object programming.

comments