Senior Staff Engineer - (Linux,AWS,Terraform,Big Data)

Experian Information Solutions

پاکستان

درخواست بھیجیں

جولائی ۰۹, ۲۰۲۵ شائع ہوئی 19 نے دیکھا Report Job

The ideal candidate will bring over five years of experience in Linux, networking, and security fundamentals, combined with a strong background in AWS cloud platform and infrastructure. This role demands expertise in infrastructure as code using tools like Terraform or Ansible, alongside hands-on experience managing large-scale Big Data clusters in production environments such as Cloudera, Hortonworks, or EMR. A solid understanding of observability tools including Prometheus, InfluxDB, Dynatrace, Grafana, and Splunk is essential for effective monitoring of Big Data platforms.

Key Responsibilities
- Manage and maintain large-scale Big Data clusters to ensure high availability and optimal performance.
- Implement and optimize Hadoop Distributed File System (HDFS) and Hadoop YARN environments.
- Work extensively with Hadoop file formats such as ORC, Parquet, and Avro.
- Support and enhance compute engines including Hive (Tez), Hive LLAP, Presto, and Spark.
- Analyze and improve the performance of complex SQL queries on Hive and Spark platforms.
- Develop and maintain Spark applications using Python (PySpark) and R (SparklyR, SparkR).
- Automate workflows and processes by writing scripts in Shell, Python, or other scripting languages.
- Collaborate closely with Data Analysts and Data Scientists, supporting analytical applications like SAS, R-Studio, JupyterHub, and H2O.
- Review and understand codebases in Java, Python, R, and Scala, with proficiency in at least one scripting language.
- Report directly to Senior Management, providing regular updates and insights on infrastructure and platform health.

Required Qualifications
- Bachelor of Engineering or an equivalent bachelor’s degree.
- Minimum of five years’ experience with Linux, networking, and security fundamentals.
- Proven expertise with AWS cloud infrastructure.
- At least five years of experience using infrastructure as code tools such as Terraform or Ansible.
- Hands-on experience managing production Big Data clusters (Cloudera, Hortonworks, EMR).
- Strong knowledge of Hadoop ecosystem components including HDFS, YARN, Hive, Presto, and Spark.
- Proficient scripting skills in Shell or Python.
- Ability to collaborate effectively with data teams and understand analytical tools and languages.

Preferred Qualifications and Benefits
- Experience with containerization technologies such as Docker and orchestration platforms like Kubernetes or EKS.
- Familiarity with workflow management tools including Airflow and Oozie.
- Knowledge of analytical libraries such as Pandas, NumPy, SciPy, and PyTorch.
- Experience with automation and CI/CD tools like Packer, Chef, or Jenkins.
- Prior exposure to Active Directory and Windows OS-based VDI platforms such as Citrix or AWS Workspaces.

Experian is a global leader in data and technology, empowering businesses and individuals worldwide through innovative solutions across lending, fraud prevention, healthcare, marketing, and automotive sectors. With a workforce of 22,500 employees in 32 countries and headquarters in Dublin, Ireland, Experian invests significantly in its people and advanced technologies to unlock the power of data.

The company fosters a culture of diversity, equity, and inclusion, encouraging employees to bring their authentic selves to work. Recognized as a Great Place to Work™ in 24 countries and listed among FORTUNE’s Best Companies to Work For, Experian prioritizes work-life balance, wellness, professional development, and rewarding experiences.

Employees benefit from comprehensive family well-being programs, enhanced medical coverage, and paid time off to support health and safety. Experian is proud to be an Equal Opportunity and Affirmative Action employer, committed to innovation and inclusive success.

Join Experian and contribute to creating a better tomorrow together.