Subscribe to DSC Newsletter
Nikita Shanker
  • Female
  • Phoenix,Arizona
  • United States
Share on Facebook
Share

Nikita Shanker's Discussions

How does hive insert into with select work at memory and storage level

Started this discussion. Last reply by Johan THim Feb 12. 2 Replies

I am trying to understand how does hive insert into with select works at memory and storage level.One of my oozie workflow fails due to GC overhead issue.Our query is: insert into table <>…Continue

Tags: heap, memory, overhead, GC, hive

Gifts Received

Gift

Nikita Shanker has not received any gifts yet

Give a Gift

 

Nikita Shanker's Page

Latest Activity

Johan THim replied to Nikita Shanker's discussion How does hive insert into with select work at memory and storage level
"Follow. I have the similar issue"
Feb 12
Lance Norskog replied to Nikita Shanker's discussion How does hive insert into with select work at memory and storage level
"What is the table format? Different table formats require different amounts of memory to create the output HDFS file. Text and Sequencefiles need very little memory. Hive copies data from table2 to a local files on some of the servers, then copies…"
Feb 7
Nikita Shanker's discussion was featured

How does hive insert into with select work at memory and storage level

I am trying to understand how does hive insert into with select works at memory and storage level.One of my oozie workflow fails due to GC overhead issue.Our query is: insert into table <> select a,b,c,d from table2.Now the map reduce job finished and then we get the message :Ended job = jobid: <>Loading data to table table2 Failed with exception java.lang.outofmemoryError: GC overhead limit exceeded.Failed: execution error, return code 1 from…See More
Feb 5

Profile Information

Short Bio
Graduate student in Big Data Systems. Currently working on transforming relational Database, Minibase to a Graph database. Enthusiastic of Data Mining advancements, new predictive models for data classification and extractions.
My Web Site Or LinkedIn Profile
http://https://www.linkedin.com/in/nikita-shanker-b932a430
Professional Status
Student
Years of Experience:
3.7
Your Job Title:
Graduate Student in Masters in Computer Science- Big Data Systems
How did you find out about DataScienceCentral?
LinkedIn
Interests:
Finding a new position, Networking, New venture

Comment Wall

You need to be a member of Data Science Central to add comments!

Join Data Science Central

  • No comments yet!
 
 
 

Videos

  • Add Videos
  • View All

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service