Software Engineer

Location: Santa Clara, CA

Employment Type: Regular full time

Company:  Big Data Federation uses machine learning to predict financial and economic performance for our investment fund and to advise Fortune 100 companies.  We fuel our predictions with massive amounts of interconnected data, from the company to the macroeconomy level.  Our AI prediction tool identifies latent macroeconomic risks, industry inflections, and fundamentals investment opportunities.  We are a venture-backed team of technologists, mathematicians, economists, data scientists, and programmers.

Job Profile: You will join an experienced and diverse team of software engineers and data scientists as a key contributor to our predictive analytics platform, where you will be responsible for ingesting new data sources into our data platform: this entails creating docker containers which customize the retrieval, post-processing/validation and storage of each data source into our private cloud.  You will work with data vendors and our own data scientists to optimize the ingestion process for each source: from prototype through deployment as a micro-service to our data platform.  You are encouraged to develop your full-stack engineering skills so you can help develop/maintain the data platform itself.

We particularly seek passionate, data-oriented candidates who are eager to learn new skills, and to mentor their colleagues in areas where they themselves excel.

Responsibilities include:

  • Develop/maintain data platform to ingest data and validate/store it in our cloud, and serve it up to applications through internal APIs
  • Create and maintain docker-based microservices API
  • Write data ingestion modules from external sources (Python)


  • Experience with Python, experience writing highly scalable, highly available, modular software components
  • Experience with relational databases (mysql, query optimization, troubleshooting)
  • Experience using Docker, containerized deployment
  • Experience creating RESTful APIs and micro-services
  • Experience with AWS/ECS services and other AWS stack is a plus
  • Agile development, continuous integration and test-driven design a plus
  • Knowledge of English spoken and written at medium level is required


  • Competitive salary; Stock options; Health benefits: medical, vision, and dental; Paid time off; Life insurance; Short- and long-term disability.

Please send cover letter and resume to