Subscribe to DSC Newsletter
Phil King
  • Male
  • Birmingham
  • United Kingdom
Share on Facebook
Share

Phil King's Discussions

Open Source automated ETL of web-based data sources

Started this discussion. Last reply by Phil King Sep 14. 8 Replies

Hi,I've been involved in bringing together a number of healthcare-related datasets into a bespoke online analytics service for several years.  Whilst the high volume, patient level data is handled in…Continue

Tags: linux, automation, csv, ETL

Gifts Received

Gift

Phil King has not received any gifts yet

Give a Gift

 

Phil King's Page

Latest Activity

Phil King replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Just a quick update - had a whistle-stop demo of Wrangler Pro and it does look pretty impressive.  The downside is that they want about 5000 USD per user per year plus 10000 USD server costs per annum.  Blows it out of the water for us…"
Sep 14
Phil King replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Hi Phil, Have taken a brief look at Trifacta Wrangler free version including some of the training videos.  Seems that it only works on local files when importing - so not able to grab directly from the publisher's website.  Not…"
Aug 28
Phil King replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Have signed up for a trial of Wrangler - will let you know how it goes Phil. Thanks."
Aug 24
Phil Hummel replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Just starting to investigate this company.  See what you think https://www.trifacta.com/products/wrangler-editions/"
Aug 24
Phil King replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Thank you for those three responses - I will check them out. If I were to relax my Open Source requirement and ask for recommendations including commercial tools, what are the best to check out?"
Aug 23
George Fraser replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Airflow is probably the go-to open source tool in this space."
Aug 23
Carlos Kassab replied to Phil King's discussion Open Source automated ETL of web-based data sources
"Hello, EplSite ETL might help you: https://eplsite-etl.blogspot.com - Open Source.- Easy to use.- Low resource consuming.- Just the necessary tools to do the job.- Web interface.- Very easy to customize it because it is developed in Perl, all…"
Aug 23
Jim Lola replied to Phil King's discussion Open Source automated ETL of web-based data sources
"As part of a Big Data Analytics system we developed a few years ago (called LM Ensemble/LM Wisdom), we took part of the UI-ETL component and Open Sourced it.  It is still available as Open Source on GitHub.  The UI-ETL component was…"
Aug 23
Phil King's discussion was featured

Open Source automated ETL of web-based data sources

Hi,I've been involved in bringing together a number of healthcare-related datasets into a bespoke online analytics service for several years.  Whilst the high volume, patient level data is handled in a pretty much fully automated process, an increasing number of datasets published by public bodies are also of interest to users of the service.The datasets are typically updated on a monthly, quarterly or annual basis and I currently download them when my Outlook calendar the data might be due,…See More
Aug 23
Phil King posted a discussion

Open Source automated ETL of web-based data sources

Hi,I've been involved in bringing together a number of healthcare-related datasets into a bespoke online analytics service for several years.  Whilst the high volume, patient level data is handled in a pretty much fully automated process, an increasing number of datasets published by public bodies are also of interest to users of the service.The datasets are typically updated on a monthly, quarterly or annual basis and I currently download them when my Outlook calendar the data might be due,…See More
Aug 23
Phil King updated their profile
Aug 23

Profile Information

Short Bio
30 years in healthcare data. Looking to share experience and learn new things!
My Web Site Or LinkedIn Profile
http://https://www.linkedin.com/in/thephilking/
Field of Expertise
Data Science, Business Analytics
Professional Status
Technical
Years of Experience:
30
Your Company:
QIdata
Industry:
Healthcare
Your Job Title:
Head of Informatics
How did you find out about DataScienceCentral?
Google
Interests:
Contributing, Networking

Comment Wall

You need to be a member of Data Science Central to add comments!

Join Data Science Central

  • No comments yet!
 
 
 

Follow Us

Resources

© 2018   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service