stevenwu

Right. The author finds 100% of the time for his current dataset but makes a statement that implies some certainty or inference on future cases. Like taking 100 men, 100 women and finding that 100 randomly matched pairs had the man taller than the woman 100 times, and making the claim that there is a 100% chance that men are taller than women.

The more I type the more I realize how pedantic this is, but we're emphasized in stats to pay extra attention to the conclusions we draw from the data we analyze.

stevenwu

9 lat temudiscuss

Question to spark discussion and for me to fill potential gaps in my knowledge, not to criticize the article as I very much appreciate the transparency and build-up from the simple naive initial approach to the final approach used in production:

Is anyone else bothered by the claim that there "is a 100% chance that the new version is better than the current one" shown by using bootstrap? Maybe I've just never come across such a use of bootstrap through my encounters with statistics. I know it as a tool for resampling from a population to build up properties of your estimator (mean, variance, what have you) when all you have is a dataset and no clue about the actual distribution. When I saw bootstrap with that probabilistic claim, I thought the author would calculate a bootstrapped (100-x)% confidence interval for both the current and the new weights: and if the intervals didn't overlap with one another then you can claim with (100-x)% certainty that one is better than the other. But the author creates a new statistic that is a function of both datasets; Z_i = 1 if new is better than current on iteration i (on a random subset of data) else 0, and for all N=10000 iterations Z_i = 1. The chance/probabilistic claim made of new being better than current is based on the fact that no variation was seen on Z_i (I'm also kind of skeptical that out of so many iterations with random subsets that each time the new weights were better than the current). I think at most you can say that you simulated subsets of the data and 100% of the time new > current; the current claim leads me to believe there's inference that isn't there.

Maybe I should just ask one of my past stats profs. Open to someone enlightening me.

stevenwu

9 lat temudiscuss

This is probably a good place to start: https://stripe.com/blog/teaching-employees-to-code

Blog was posted early enough to have a high likelihood of response if you ping the author with follow up questions

stevenwu

9 lat temudiscuss

Sort of related but not about the main site being down: search crashes way more often than I expected it would for such a popular site. I read an AMA by the CEO that said they're rolling out a new search by (the end of?) this summer. Probably not enough resources to scale up as fast as they want to.

stevenwu

9 lat temudiscuss

The honesty in this post is really refreshing and insightful. There are a lot of people in tech who would kill to be in your position (see every monthly-ish thread about how to make passive income or how to make money off of an app). It's interesting to think about what proportion of those such people who achieve their goal would end up feeling like how you feel - having a lot of free time, and spending a large chunk of it worrying instead of being as hyper-productive as they imagined they could be.

stevenwu

9 lat temudiscuss

On the topic of raising/burning $:

I remember seeing that they had a position open for a software engineering role. If I remember the job description correctly, they've built out their own online store?

I recently saw that you can buy their products through Amazon - I wonder if it was money well spent to roll out their own web store versus using Amazon/Shopify. As a past customer I don't remember seeing any particularly unique feature that made hiring in-house staff for this aspect necessary.

stevenwu

9 lat temudiscuss

It's now the end of my engineering and I am now occupied with the thoughts of what my life had been if I would have chosen the road not taken.

Isn't it equally as likely that in the alternate universe where you pursued Physics that you would have either been unhappy with the realities of a job in the field (maybe too much scut work, lab work, what have you) or even been facing unemployment?

I ask this because I know a lot of people on the other side of the green grass (people who pursued something not as employable as CS for reasons of passion and have displayed regret about not diving into technology for the ubiquity of it in our present day-to-day)

stevenwu

9 lat temudiscuss

I'm OK with these kinds of quirky add-on allowances so long as the category for the add-on is positive, either for you as a person (e.g.: fitness, learning) or the local community (e.g.: paid time off volunteering, charity matching). We'd all be better off if we were a little more involved with our community, a little more active, and eating a little bit healthier, and the most common excuse for not doing these is "I don't have the time/money." If you just give extra cash how can you achieve the same effect in encouraging people to live healthier?

stevenwu

9 lat temudiscuss

Community Supported Agriculture (CSA) Allowance

Fresh, organic, local fruits and vegetables are great to have on-hand, so we'd also like to offer a CSA membership to you. Finding a local provider will be up to you, but you can expense the cost of a seasonal or annual CSA share as an employee benefit, up to $1,000/year.

Count that as a perk I've never heard of before.

Cool peek into how a company that works remotely can function successfully. This handbook could be a blueprint for that model - are there any other examples of remote companies publishing their internal handbooks?

stevenwu

9 lat temudiscuss

I was in a similar spot. Studied Math/Stats, and my friends were not in tech. I've made some now, but still don't have many really good friends in tech. It seems studying CS can be a real group activity, where you work together on assignments/apps/job prep. I missed out on that - just finished a job search & prep that was completely solo after working on apps completely solo.

Going to local meetups helps but it's probably leaving you unsatisfied due to the low frequency. Some suggestions that worked for me:

  - Keep an eye out for local tech companies you want to work for or just like: some might host open houses, or sponsor hackathons. You may hear about this through Meetup.com but not always
  - I'm betting Chicago has a Slack dev group. I'm in the Slack dev group for where I am now and where I'm going to move, and it's a good way to stay connected, ask questions, and absorb discussion that you want but aren't getting in your day-to-day
  - If you have any interest in volunteering, look for things like Ladies Learning Code and the like going on in your area. Usually the mentors there who want to help young kids are established in the industry. You can go, help younger generations with what you know, with an added benefit of getting to know other mentors, who may help you down the line.

This is a phase in your life that will pass once you start working in the field. I know it may feel hard right now, but hopefully this piece of advice helps (which I found true for me, and I hear from others in the industry): your "outsider"-ness can be spun into a benefit. You see the tech world through different eyes, you come unhinged without biases. And, being fresh and without experience, you will be hungrier, more persistent, and more willing to learn something new than somebody who has roots and comfort zones.

Once you pass through this phase and "make it", don't forget to give back when called upon; you will now be "that" person in this network comprised of non-tech people, in a world that is ubiquitous with tech.

stevenwu

9 lat temudiscuss

You only mentioned Foursquare as one of the companies you've applied to, so without more info from you I'm going to take a shot in the dark here: are you only applying for those types of companies that are very well known? It's pretty rare for your first job out of education (whether it's a degree, or a bootcamp) to be "SWE at <Company Your Friends Know>". There's a whole world of companies out there that aren't as sexy but can be amazing stepping stones to the job you want.

To add personal experience to this comment, I've worked at two places before landing such a "dream" job - a < 50 person startup, and a company with businesses around its satellites that employs ~5000 people but I guarantee your average citizen hasn't heard of it. Some former coworkers at that startup were there as their first job, and went on to work for Google/Palantir/themselves. These less sexy companies and startups are waiting for great applicants.

stevenwu

9 lat temudiscuss

You are right to think that there is a correlation between previous salaries and difficulty in finding jobs. In an ideal world, if your true worth is $X/year but you're willing to work for $(0.90X)/year, then you should be able to nab a job right? After all, you're a bargain. I struggled with this myself as a student job hunting, fighting the mentality of self-worth. All my jobs previous to my new one were low pay for our industry (highest was $25/hr CAD). As someone who's had to learn to embrace this mental shift when negotiating with much better offers relative to what I was used to, the first thing I will say is that you are spot on: a race to the bottom helps no one.

So OK, let's talk about your main question: where to begin to command interviews and ultimately a better salary? The advice here so far has been standard: (a) look outside of one area (b) do side projects and (c) talk to recruiters/network. This is fine advice, but IMO, not the most helpful. Advice like "find your niche" is better... but that is a difficult task on its own, to decide on one path that is by definition obscure.

Again I will draw from personal experience to try and share what I've learned. What worked most for me was to aim for a dream job, and to do things that would lead to building skills that would make me qualified for this dream job. I was in my 3rd year of undergrad - at the time, it was to work in an NBA front office - I was studying stats, and didn't have many interests besides having fun and following basketball. In following this pursuit to its limit, I learned many skills along the way that I didn't have until I decided what I most wanted: web scraping, web development, databases, scripting, algorithm development, computational statistics, explaining statistics to non-technical users, trying to build a business, etc. Side projects, building a network (both local and out of town), and finding my niche (data science + web dev + product skills) resulted from this goal. They would have never happened on my own initiative - I would have been one of many people dreaming of app ideas and side projects to start but never finish. This has a nice benefit that comes for free: you are now different than the rest. Your perspective to solving problems and your experiences will be unique (to what degree, matters based on whatever it is you choose).

The downside of my advice is that it is not a short fix: myself, I am now 3-4 years from when I started this story, as I am soon to graduate from my MSc Stats. But you can accelerate this process, and I can tell you it works: applying to jobs coming out of my undergrad, I only heard from companies that you wouldn't recognize. This year, I got on-sites at Amazon and Capital One before deciding to accept a dream job offer at Shopify. You can do it - and I hope that this is an answer and story that can help you in your search of where to begin your change in approach.

P.S., when you get there, do not underestimate the truth behind how useful resources like CTCI, whiteboarding, etc. are for the interview stage.

stevenwu

9 lat temudiscuss

You are a programmer.

stevenwu

9 lat temudiscuss

Other than the mentioned advice of going to office hours with a purpose, I would say a golden way to get your professor's attention is to read their papers (select the top k most cited/interesting/whatever heuristic works for you). After doing that, you will undoubtably have focused questions to ask that can lead into discussion that will implicitly signal to the professor your interest/curiosity/passion (don't bog yourself down in the low level details). Reading papers is good practice anyway if (a) stuff like independent study intrigues you and (b) you want to stay current within your field of interest.

stevenwu

9 lat temudiscuss

The answer to your question all depends on exactly what kind of jobs you are even remotely interested in?

In general it's best to compound your current skills and not go in a completely perpendicular direction to where you are going now.

Given that: - you are good in JavaScript - you haven't provided much filter of criteria - you mentioned a curiosity in machine learning - you care about job prospects and want a skill that will stay relevant

Have you considered data visualization? If you become a master at that, you can break into the higher level parts of machine learning/data science that require visualizing results to end-users/interested stakeholders. I'm good enough with D3.js to understand examples and create my own modified graphs, but I wish I had the skills to recreate all of the cool visuals you can see in their gallery.

stevenwu

9 lat temudiscuss

I am only a mere MSc Statistics student in a Canadian university, but from my perspective, if you are succeeding as a physics PhD in a tough and respected program, then you are more than equipped to do anything you set your mind to in the tech sphere. I'm interested in AI/ML too and I read papers to try and keep up - I'm surprised at how many ideas and methods have ties to statistical physics and other domains that you're probably already comfortable with as a physicist.

In my city's (Vancouver) Slack chat, a lot of the most knowledgable participants in data science discussions are physicists (and they're proud of the fact that they're physicists). Check out the Director of Data Science at Unbounce, who published papers and worked as a postdoc as a physicist before starting in industry data science at PlentyOfFish. Worst case scenario if you find yourself pigeonholed, find a home in Vancouver with like-minded people.

I'd say finish your PhD, skip the AI PhD and go for your ultimate goal ASAP!

stevenwu

9 lat temudiscuss

I don't agree with this article mainly because one's passion for anything in life is dynamic. It starts and stops at points in your life that you can't anticipate.

I didn't know anything about programming until university - I took the courses CS majors had to take in my stats degree until the end of 2nd year, and I definitely wasn't "passionate" about it. Actually, I was quite convinced I didn't want programming to be in my career whatsoever. But after my first co-op term in a start-up where I got to see what actually could be done with a team of people who knew how to code, coupled with 3rd year classes being more interesting and relevant than the fundamentals, I gained my passion. I now run a SaaS that combines programming, statistics, and other passions.

If I read your article after 2nd year while applying for my first jobs and taken this advice, I would have said "screw it, life's too short" and not be in the situation now where so many more doors are open compared to my classmates. I'd be on track for pigeon holing myself into boring stats jobs that need SAS certifications, with none of the skills I have now for a career in modern data analysis in the tech industry.

Side note, I hate reading about developers who lament about the career trajectory compared to being a doctor/lawyer. Most good developers are smart enough to genuinely believe that being a doctor would have been attainable if you just switched another lever earlier in life. I think this is a valid regret if you had actually always wanted to dedicate your life to legally/medically advising people, but you chose programming for the money. But for almost all of these developers, the decision to choose their career wasn't 100% about money and being a doctor/lawyer was never a serious consideration past wishful thinking. You're just viewing the grass as greener on the other side without considering the cons and consequences of actually being a doctor/lawyer, and that type of negative mentality is not healthy.

stevenwu

10 lat temudiscuss

Tangentially related, does anyone here use their APIs? I started playing around with their Earth API (https://api.nasa.gov/api.html#earth) but ran into a bug that turns out to have been reported half a year ago.

https://github.com/nasa/api-docs/issues/63

I sent an e-mail to their Contact Us mailbox with no reply. I'm kind of hoping someone who works there sees this, because getting an image larger than the default size would be really cool.

stevenwu

10 lat temudiscuss

Hey, thanks for letting me know - I thought filling in the e-mail field made it visible to everyone on my profile. Sending you an e-mail now.

stevenwu

10 lat temudiscuss

Anytime I find myself thinking like this, I use this reminder: the only thing separating myself from being one of those people is actually doing it (whatever "it" is). It's psychological: until you internally want to write (picking one example from your list of wants) more than you want to read or browse social media, you will keep doing the same thing you've been doing.

Think about it this way: in your case, you have it easy, in that you have the skills to be doing what it is you want to do. I have plenty of non technical friends that want to make an app, but at the moment they're not capable. You're equipped to achieve your goals, so harness that internal motivation and act on it. Write instead of reading. Be a social media presence instead of following social media.

>> Honestly i think i'll only be contempt when i cut off everything non work related from my life to be more productive

FWIW I've had those thoughts, and it was a factor in my decision to leave home where I was comfortable with great friends. IME I am more productive as a whole because a big chunk of social time has gone into work time but I feel like I do less work per hour, if that makes sense. Everyone is different and I think you owe it to yourself to find that out (either you end up truly content or you realize what is important in your life, both of which are great destinations to find yourself in)

Submissions

Ask HN: Companies who want to outsource some analytics to a data science class?

Ask HN: Approach to pricing? Launching for the first time to targeted users

comments