Software Engineer (python, backend)

Location: Santa Clara, CA

Employment Type: Regular full time

Company:

Big Data Federation, Inc. is one of a new breed of asset managers on a quest to reinvent the investment process. We are unburdening humans of the manual job of making sense of endlessly growing volumes of data, and passing this responsibility onto machines. We believe our automated processes bring significant efficiencies in uncovering company fundamentals and driving fund outperformance.

We live by the motto “Nothing is random”.

Our home grown, scalable, and smart data platform continuously processes and interconnects tens of billions of data records daily from a diverse and growing number of publicly-available data sources. We are a team of technologists, mathematicians, economists, data scientists, and programmers backed up by one of the leading venture capital firms.

Job Profile: You will join an experienced and diverse team of software engineers and data scientists as a key contributor to our predictive analytics platform, where you will be responsible for ingesting new data sources into our data platform: this entails creating docker containers which customize the retrieval, post-processing/validation and storage of each data source into our private cloud.  You will work with data vendors and our own data scientists to optimize the ingestion process for each source: from prototype through deployment as a micro-service to our data platform.  You are encouraged to develop your full-stack engineering skills so you can help develop/maintain the data platform itself.

We particularly seek passionate, data-oriented candidates who are eager to learn new skills, and to mentor their colleagues in areas where they themselves excel.

Specific Responsibilities to Include:

  • Cloud Infrastructure planning, testing, development and deployment,
  • Create and deploy high availability solutions for Big Data driven services
  • Create various backend micro-services, maintain existing one, actively participate in the platform development process.
  • Install, configure manage and extend a variety of software systems including software for version control, continuous integration, continuous deployment, performance data collection and analysis, systems monitoring and more
  • Automate the compilation, continuous integration, testing, packaging, and distribution of multiple software components
  • Monitor and improve operations performance, security and stability
  • Investigate failures, fix them and execute root-cause analysis
  • Engineer systems and tools to support the build, integration and verification of complex software systems
  • Work closely with developers to tailor tools and solutions for rapidly evolving needs
  • Continuously discover, evaluate and implement new technologies or services to maximize operations efficiency and reliability

Experience

  • Demonstrated success deploying and scaling high traffic services in AWS (AMI’s, EC2, S3, CloudWatch, ECS, VPC, RedShift, Network Routing/ACL’s, ELB)
  • Expert networking knowledge and ability to maintain various server web server software (NGINX, OpenResty, Apache etc.)
  • Experience implementing and managing CI/CD with Jenkins, Docker, AWS Fargate and AWS ECS clusters
  • Experience Python3 (django/flask), micro-services development, data-bases (mysql) is absolutely a must.
  • Extremely comfortable with Linux, shell-scripting, and git
  • Experience with Kubernetes and Google Cloud would be a plus
  • Experiences with IAC tool such Terraform or Ansible would be a plus
  • Experience with other languages: R, JS, Go, Lua, Ruby would be a plus
  • Thrive in a startup/fast-paced environment
  • Excellent oral and written communication skills

Qualifications

  • Bachelor’s degree in computer science or related field, or equivalent experience
  • 4+ years of experience

Compensation

  • Competitive salary; Stock options; Health benefits: medical, vision, and dental; Paid time off; Life insurance; Short- and long-term disability.

Please send cover letter and resume to jobs@bigdatafed.com