Improbable Defence is hiring a mid-senior Data engineer. Excitingly, this is a brand new team so its remit is still being determined, but may well include:

• Storing and cataloguing data used in simulations (models, assets, configuration, etc) in a secure way.
• Versioning data, managing metadata, data classification
• Allowing data inputs to be queried to help our users rationalise about simulation outcomes
• Building scalable data pipelines and tools to generate new inputs and models from a variety of (often real-world) data source
• Understanding where data originated and where it is used.

Areas for Impact

• Contribute alongside the team to design and develop the framework and tools that enable applied scientists to build pipelines to extract, transform, synthesise and integrate data from various sources.
• Build tools for the automation and scheduling of data extraction from both static and live sources.
• Help to design and build the services needed for accessing and updating the data and models for use within a synthetic environment.
• Build using standard data science tooling where possible to provide an intuitive experience for users with minimal onboarding time.
• Engineer systems for scalability and efficiency so that our customers can process large volumes of data in a timely way.
• Build secure and auditable systems.

We'd like to hear from you if you identify with the following:

• You are an experienced Data Engineer with experience building relevant systems.
• You have experience building with scalable data processing implementations using multiple technologies (eg Hadoop, Apache Spark, Apache Airflow, Kafka, Mongo, Timescale, MinIO) and across multiple platforms (AWS, GCP, Azure, possibly custom environments too).
• You are pragmatic and able to identify the most impactful work among competing requirements.
• Ideally, you have knowledge of Go, Python or C++, but we’re open-minded.

About Us

Improbable is determined to foster an environment where people can do their best work and feel like they belong. We believe a healthy culture, strong values and contribution from a diverse range of individuals will help us to achieve success.

We do not discriminate based on race, ethnicity, gender, ancestry, national origin, religion, sex, sexual orientation, gender identity, age disability, veteran status, genetic information, marital status or any other legally protected status.