19 days old

Software Development Engineer – Big Data, AWS Elastic MapReduce (EMR) - Amazon

New York, New York
  • Jobs Rated

Job Description

Want to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!

Amazon EMR is a web service which enables customers to run massive clusters with distributed big data frameworks like Apache Hadoop, Hive, Tez, Flink, Spark, Presto, HBase and more, with the ability to effortlessly scale up and down as needed. We run large number of customer clusters, enabling processing on vast datasets.

We are developing innovative new features including our next-generation cluster management system, improvements for real-time processing of big data, and ways to enable customers to more easily interact with their data. We’re looking for top engineers to build them from the ground up.

Here are sample features that we have recently delivered

  • Authenticate with Kerberos: https://aws.amazon.com/about-aws/whats-new/2017/11/now-enable-kerberos-authentication-and-emrfs-authorization-in-amazon-emr/
  • EMR Per Second Billing: https://aws.amazon.com/about-aws/whats-new/2017/10/amazon-emr-now-supports-per-second-billing/
  • Spark Application History in EMR Console: https://aws.amazon.com/about-aws/whats-new/2017/09/now-view-apache-spark-application-history-and-yarn-application-status-in-the-amazon-emr-console/
  • Custom AMIs: https://aws.amazon.com/blogs/big-data/create-custom-amis-and-push-updates-to-a-running-amazon-emr-cluster-using-amazon-ec2-systems-manager/
  • EMR Instance Fleets: https://aws.amazon.com/blogs/aws/new-amazon-emr-instance-fleets/
  • Auto Scaling EMR Clusters: https://aws.amazon.com/blogs/big-data/dynamically-scale-applications-on-amazon-emr-with-auto-scaling/
  • EMR Security Configurations: https://aws.amazon.com/blogs/big-data/encrypt-data-at-rest-and-in-flight-on-amazon-emr-with-security-configurations/
  • Responding To State Changes with AWS CloudWatch Events: https://aws.amazon.com/blogs/big-data/respond-to-state-changes-on-amazon-emr-clusters-with-amazon-cloudwatch-events/

This is a hands-on position where you will be do everything from designing and building extremely stable components and cutting-edge features for the savviest customers in the business to help them get the best results.

You will have a chance to work with the open source community and contribute significant portions its software to open source projects possibly including Apache Hadoop, Spark, Pig and Hbase. You need to not only be a top software developer with excellent programming skills, an understanding of big data and parallelization, and a stellar record of delivery but also excel at leadership and customer obsession and have a real passion for massive-scale computing. If you want to truly test your mettle against the hardest challenges in distributed systems to build solutions for large scale problems in a wide variety of domains, come join our group.

Your responsibilities will include

  • Translation of complex functional and technical requirements into detailed architecture and design
  • Deliver systems and features with top-notch quality, on time
  • Develop new technologies for monitoring production clusters
  • Own the software development process end-to-end, including: working with engineers and product managers to develop requirements; designing, architecting, planning, implementing, and testing new systems and features; deploying, and operating the production EMR systems.

In joining our team, you will get to work with a minimum of technical supervision, while playing a variety of roles as needed to respond efficiently to multiple program priorities. You will get to collaborate with some of the best and brightest minds in the industry. You'll enjoy a competitive salary, great benefits, a creative and agile work environment, and the exciting opportunity to be part of a fast-paced and growing team and one of the most innovative technology companies - but most of all, you will get the satisfaction of making products that millions use everyday to great effect!

For more information

  • AWS re:Invent 2017 Amazon EMR: https://www.youtube.com/watch?v=1CAWf9VDgFM
  • AWS re:Invent 2017 Keynote - Andy Jassy: https://www.youtube.com/watch?v=1IxDLeFQKPk
  • AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
  • AWS EMR: https://aws.amazon.com/emr/


Basic Qualifications

  • Master's degree in Computer Science or equivalent with 5+ years of experience OR Bachelor's Degree in Computer Science or equivalent with 7+ years experience, in: software development, including design, implementation, debugging, and support
  • Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
  • Strong proficiency in developing objected-oriented software, with deep experience in one or more relevant languages (Java, C, C++, C#)

Preferred Qualifications

  • Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
  • Experience designing and building highly-scaled distributed systems and web services
  • Thorough understanding of parallel algorithms, concurrency, asynchronous architectures
  • Experience working in an agile software development organization
  • Experience building with SOA using Java on Linux
  • Proficiency in high-performance, multi-threaded programming
  • Knowledge of and contribution to Hadoop ecosystem
  • Experience with distributed systems architecture
  • Experience with one or more of Ruby, Python, Perl
  • Proven ability to effectively drive cross-team solutions that may have complex dependencies
  • Masters in Computer Science with emphasis on distributed systems and Big Data architectures is a plus.

Company - Amazon.com Services, Inc.
Job ID: A694284


Jobs Rated Reports for Software Developer

Syracuse Analytics Online
Take your analytical skills to the next level. Earn an M.S. in Business Analytics online. GMAT waivers available. Apply now!

Featured Jobs

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

Software Development Engineer – Big Data, AWS Elastic MapReduce (EMR) - Amazon

Amazon Web Services
New York, New York

Join us to start saving your Favorite Jobs!

Sign In Create Account
Software Developer
11th2018 - Software Developer
Overall Rating: 11/220
Median Salary: $102,280

Work Environment
Very Low
Very Good
Powered ByCareerCast