Data Engineer Job at Novel Technology Services, San Francisco, CA

MkUweHVlZVdXNDJWMzVzNnpqMklwMjJ6Q1E9PQ==
  • Novel Technology Services
  • San Francisco, CA

Job Description

Before reading the Job description please note:

  • This position is a hybrid role and candidates must be currently located in the San Francisco Bay area.
  • The client requires candidates to possess five years of experience as a Data Engineer. While internships are valuable, they will be considered supplementary to the required five years of experience.
  • Recent work in a startup organization

Data Engineer:

Our client is Delivering the most comprehensive identity insights, our client’s platform equips businesses with fully automated KYB (Know your Business) solutions for Risk, and Fraud management, setting new standards in business verification.

The solution is designed for FS institutions to emphasize their interest in a coordinated effort to mitigate B2B fraud, reduce the risk associated with working with small businesses, and create a centralized, privacy-compliant entity for data-sharing between financial institutions.

Looking for a data engineer to assist with data collection across a variety of fragmented data sources available from both government, public, and private databases.

Responsibilities:

  • Designing the database schema
  • Migrating production tables
  • Working with large scale data in the terabyte range
  • Maintaining operational efficiency of database
  • Normalizing disparate schemas to a single unified scheme
  • Abstracting reusable components 
  • Strength in minimizing the amount of new code for new pipelines and instead creating internal packages that allow a high level of reusability 

Requirements

Experience with:

  • ETL (extract, transform, load)
  • Database design
  • Primary key, foreign key
  • Indexing
  • Partitioning
  • Access patterns
  • Migrations
  • Data pipelining
  • Core data concepts: ACID transactions, Idempotency, Orchestration

Technologies:

  • Airflow
  • Google Cloud Platform (GCP)
  • GCP Dataflow (aka Apache Beam)
  • PostgreSQL
  • Python
  • Pydantic (a python library)
  • Distributed systems

Job Tags

Full time,

Similar Jobs

Interim HealthCare of Augusta

Home Care Nurse - RN LPN Job at Interim HealthCare of Augusta

Over worked, unappreciated & underpaid, forced to provide patient care in a way that doesnt focus on care or the patient all in...  ...HealthCare of Augusta our #1 priority is doing right by our Clients; our Nurses and the Patients they care for.Come see what makes us... 

Ardent Health

Registered Nurse / RN Four Seasons OBGYN Job at Ardent Health

Overview Join our team as a day shift, full-time, Obstetrics/Gynecology (OBGYN) Registered Nurse (RN) in Tyler, TX. Why Join Us? Thrive in a People-First Environment and Make Healthcare Better Thrive: We empower our team with career growth opportunities...

Evlv

Pattern Maker Job at Evlv

 ...Title : Pattern Maker Industry: Fashion and Apparel Location: Montreal, QC Position Status : Full-time, permanent (onsite) Working Hours: Monday - Friday Salary: 75k-80k The right candidate has the following soft skills: Proactive Detail-oriented... 

The Eye Group

Join Ophthalmology practice in Brooklyn, NY Job at The Eye Group

Full time opportunity to join this multispecialty ophthalmology practice. Work side by side with MDs providing comprehensive eye care & pediatric exams. Prefer residency trained or candidates with a strong ocular disease or pediatrics background. Full time position with... 

Florida Conference of Seventh-day Adventists

Music Teacher Job at Florida Conference of Seventh-day Adventists

Job Summary: The position of Music Teacher will report to the Principal and is responsible for teaching music education to pupils in grades K-8, and assists the staff of the school in the improvement and appreciation of music instruction.Helps pupils grow in their enjoyment...