Big Data Engineer
[online recruitment]
Location: Wroclaw
Reference number: VR-52278
Project Description:

Our client, a top tier investment banks is constructing a data analytics platform which will be a one stop shop for all data consumers in credit division. It is a cloud based platform that leverages modern development practices and a secure, cloud based architecture to enable the ingestion & processing of massive data sets at scale, and the delivery of insights to clients. Platform leverages Microsoft Azure, Spark, and Tableau for data consumptions and visualization. With built in frameworks for Machine Learning, Data Quality and Real-Time, this platform will be the strategic source of data for downstream systems.

We are looking for hands-on Big Data Transformation and ETL engineers to work on a long-term project in a globally distributed Agile team. The candidate should be a proven problem solver, someone who enjoys and embraces the variety of tasks a role like this provides.

On-line on-boarding, remote work from home, flexible working schedule, as well as relocation options across Luxoft locations in Russia or Poland are available.

Responsibilities
  • Design, architect and support new and existing data and ETL pipelines and recommend improvements and modifications.
  • Create optimal data pipeline architecture and systems using Apache Airflow
  • Assemble large, complex data sets that meet functional and non-functional business requirements.
  • Be responsible for ingesting data into our data lake and providing frameworks and services for operating on that data including the use of Spark & Databricks.
  • Analyze, debug and correct issues with data pipelines
  • Operate on or build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Spark and Azure technologies.
Skills
  • 8+ years of experience building high performance scalable enterprise analytics or data centric solutions
  • At least 5 years of experience implementing complex ETL pipelines preferably in connection with Hadoop or Spark
  • Experience working with Spark data pipeline and or streaming experience
  • Expert in Python, or can demonstrate ability to readily learn new languages and affinity for them.
  • Understand the nature of distributed systems design.
  • Hands-on experience with Azure.
  • Strong understanding & usage of algorithms and data structures.
  • Work well in a team environment, and are a self-starter.
  • Ability to lead, or have led teams before.
  • Expertise building and managing python libraries
  • Experience working in Scrum teams
Our offer
Career and Professional Development
  • International projects for clients all over the world
  • Competitive salary
  • Individual development plan
  • Managerial Targeted Training programs
  • BRIDGE Mentoring Program
  • Luxoft Training Center
  • Language Classes
  • Self-learning online library
  • Global Relocation Program
  • Internal Mobility (a chance to gain experience in varied projects and technologies)
  • Professional communities for knowledge-sharing (Agile, Tech, Business)
Well-being
  • Group Life Insurance
  • Travel Insurance
  • Private Healthcare (dental care, unlimited consultations of specialist physicians)
  • Medical costs reimbursement for employees
  • Benefit Program (Cafeteria and Multisport Card)
Work-Life Balance
  • LuxGood Program (a wide range of health and well-being initiatives)
  • After-hours groups (sport, trips, board games, cultural activities)
  • Company and Team events
  • BeLux - discount offers program (banking, car leasing, other)
  • Convenient locations in modern offices
Career and Professional Development
  • International projects for clients all over the world
  • Competitive salary
  • Individual development plan
  • Managerial Targeted Training programs
  • BRIDGE Mentoring Program
  • Luxoft Training Center
  • Language Classes
  • Self-learning online library
  • Global Relocation Program
  • Internal Mobility (a chance to gain experience in varied projects and technologies)
  • Professional communities for knowledge-sharing (Agile, Tech, Business)
Well-being
  • Group Life Insurance
  • Travel Insurance
  • Private Healthcare (dental care, unlimited consultations of specialist physicians)
  • Medical costs reimbursement for employees
  • Benefit Program (Cafeteria and Multisport Card)
Work-Life Balance
  • LuxGood Program (a wide range of health and well-being initiatives)
  • After-hours groups (sport, trips, board games, cultural activities)
  • Company and Team events
  • BeLux - discount offers program (banking, car leasing, other)
  • Convenient locations in modern offices
Please send your CV with consent for processing your personal data for recruitment purposes using the below button
APPLY
Informujemy, że administratorem danych jest Luxoft z siedzibą w Krakowie, ul. Bora Komorowskiego(dalej jako "administrator"). Masz prawo do żądania dostępu do swoich danych osobowych, ich sprostowania, usunięcia lub ograniczenia przetwarzania, prawo do wniesienia sprzeciwu wobec przetwarzania, a także prawo do przenoszenia danych oraz wniesienia skargi do organu nadzorczego. Dane osobowe przetwarzane będą w celu realizacji procesu rekrutacji. Podanie danych w zakresie wynikającym z ustawy z dnia 26 czerwca 1974 r. Kodeks pracy jest obowiązkowe. W pozostałym zakresie podanie danych jest dobrowolne. Odmowa podania danych obowiązkowych może skutkować brakiem możliwości przeprowadzenia procesu rekrutacji. Administrator przetwarza dane obowiązkowe na podstawie ciążącego na nim obowiązku prawnego, zaś w zakresie danych dodatkowych podstawą przetwarzania jest zgoda. Dane osobowe będą przetwarzane do czasu zakończenia postępowania rekrutacyjnego i przez okres możliwości dochodzenia ewentualnych roszczeń, a w przypadku wyrażenia zgody na udział w przyszłych postępowaniach rekrutacyjnych - do czasu wycofania tej zgody. Zgoda na przetwarzanie danych osobowych może zostać wycofana w dowolnym momencie.
APLIKUJ TERAZ

Zobacz podobne oferty

Koordynator-analityk IT

Oponeo.pl S.A.
kujawsko-pomorskie / Bydgoszcz
Aktywne od: dzisiaj

Specjalista ds. Analiz

IT CONNECT Sp. z o.o.
mazowieckie / Warszawa
Aktywne od: wczoraj

Big Data Engineer

Luxoft
mazowieckie / Warsaw
Aktywne od: wczoraj