Here I specifically discuss the case of a PhD degree from a third-tier university, though to some extent, it also applies to master degrees. Many professionals joining companies such as Facebook, Microsoft, or Google in a role other than a programmer, typically have a PhD degree, although there are many exceptions. It is still possible to learn data science on the job, especially if you have a quantitative background (say in physics or engineering) and have experience working with serious data: see here. After all, learning Python is not that hard and can be done via data camps. What is more difficult to acquire is the analytical maturity. 

University of Namur

In my cased, I did my PhD at the University of Namur, a place that nobody has heard of. The topic of my research was computational statistics and image analysis. These were hot topics back then, and I was also lucky to work part-time in the corporate world for a state-of-the-art GIS (Geographic Information System) company, working with engineers on digital satellite images, as part of my PhD program, thanks to my mentor. Much of what I worked on is still very active these days, on a much bigger scale. It was the precursor of automated driving systems, and the math department in my alma mater was young and still very creative back then. This brings me to my first advice when choosing a PhD program.

Advice #1

• If you come from a poor background, your options might be more limited (this was my case), and you need to leverage everything you can. My parents did not have the money to send me to expensive schools, and I ended up attending the closest one to avoid spending a lot of money on rent. On the plus side, I did not accumulate student loans.
• Before deciding on a PhD program, carefully choose your mentor. Mine was not known for his research, but he was well connected to the industry, managed to get money to fund his projects, and was working on exciting, applied projects. 

A side effect on my last piece of advice is that if your goal is to stay in Academia, you may have to rely on yourself to make your research worthy of publications and susceptible to land you a tenured position. The way I did it is summarized in my next advice. You want ideally to leave all doors open, both Academia and other options.

Advice #2

• Be proactive about reaching out to well respected professors in your field. Attend conferences and meet peers from around the world. Accept jobs such as reviewers. Start publishing in third-tier journals, move to second-tier, and then get a few ones in first-tier journals before completing your PhD. The one I published in Journal of Statistical Society, Series B, is what resulted in me being accepted as a postdoc at Cambridge University. Initially when it was accepted, it only had my name on it. 
• It helps to be passionate about what you do. My very first paper was in Journal of Number Theory, during my first year as a PhD student. It happened because I had a passion for number theory that I developed during my middle-school and high-school years. I hated high-school math (repetitive, boring mechanical exercises) but loved the math that I discovered and self-learned myself during these years, mostly through reading. I was the only student to participate (and be a finalist) at the national Math Olympiads, in my school. When you are young, it's something good to have on your resume. 

So to answer the original question - does it hurt coming from a low ranking school - at this point you know that you can still succeed despite the odds. But it requires patience, perseverance, and you must be very good at what you do. Perhaps the biggest drawback is the lack of great connections that top schools offer. You have to make up for that. Also great schools have state-of-the-art equipment and labs (so you can learn the most modern stuff), but somehow my little math department didn't lack these, so I was not penalized for that. I also cultivated great relationships with the computer science department. At the end, my research was at the intersection of math, statistics and computer science.

My last piece of advice is about what happens next after completing your PhD. In my case, I started a postdoc at Cambridge then moved to the corporate world (after failing a job interview for a tenured position) and eventually became entrepreneur, VC-funded executive, and sold my last venture recently to a publicly traded company. I still do independent math research, even more so and of higher caliber than during my PhD years. 

Advice #3

• Contact other successful professionals who came from a third-tier university to ask for their advice. In my math department, two other PhD students in my cohort ended up having a stellar career: Michel Bierlaire (postdoc MIT after Namur) is now full professor at EPFL; Didier Burton (also postdoc MIT after Namur) ended up as an executive at Yahoo. 
• If you can, leverage the fact that you are very applied, don't have student loans, and thus you can ask a lower salary, be more competitive, gain various horizontal experience in many places while developing world-class expertise in a few areas.  I eventually realized that working for myself (not as consultant, but entrepreneur) was what I liked best.

You may argue that you don't need any diploma to create your own self-funded company, not even elementary school, but in the end I believe I got the best I could out of my PhD. In my case, it also implied relocating several times, from Belgium (due to lack of jobs) to UK to United States, and from the East Coast to the Bay Area and finally Seattle. I've been through various bubbles and market crashes; you may use your analytical skills to navigate them the best you can, selling and buying at the right time, understanding the markets, and emerge stronger each time. 

About the author:  Vincent Granville is a data science pioneer, mathematician, book author (Wiley), patent owner, former post-doc at Cambridge University, former VC-funded executive, with 20+ years of corporate experience including CNET, NBC, Visa, Wells Fargo, Microsoft, eBay. Vincent also founded and co-founded a few start-ups, including one with a successful exit (Data Science Central acquired by Tech Target). He is also the founder and investor in Paris Restaurant in Anacortes, WA. You can access Vincent's articles and books, here.