Data Engineer Job at Novel Technology Services, San Francisco, CA

MkUweHVlZVdXNDJWMzVzNnpqMklwMjJ6Q1E9PQ==
  • Novel Technology Services
  • San Francisco, CA

Job Description

Before reading the Job description please note:

  • This position is a hybrid role and candidates must be currently located in the San Francisco Bay area.
  • The client requires candidates to possess five years of experience as a Data Engineer. While internships are valuable, they will be considered supplementary to the required five years of experience.
  • Recent work in a startup organization

Data Engineer:

Our client is Delivering the most comprehensive identity insights, our client’s platform equips businesses with fully automated KYB (Know your Business) solutions for Risk, and Fraud management, setting new standards in business verification.

The solution is designed for FS institutions to emphasize their interest in a coordinated effort to mitigate B2B fraud, reduce the risk associated with working with small businesses, and create a centralized, privacy-compliant entity for data-sharing between financial institutions.

Looking for a data engineer to assist with data collection across a variety of fragmented data sources available from both government, public, and private databases.

Responsibilities:

  • Designing the database schema
  • Migrating production tables
  • Working with large scale data in the terabyte range
  • Maintaining operational efficiency of database
  • Normalizing disparate schemas to a single unified scheme
  • Abstracting reusable components 
  • Strength in minimizing the amount of new code for new pipelines and instead creating internal packages that allow a high level of reusability 

Requirements

Experience with:

  • ETL (extract, transform, load)
  • Database design
  • Primary key, foreign key
  • Indexing
  • Partitioning
  • Access patterns
  • Migrations
  • Data pipelining
  • Core data concepts: ACID transactions, Idempotency, Orchestration

Technologies:

  • Airflow
  • Google Cloud Platform (GCP)
  • GCP Dataflow (aka Apache Beam)
  • PostgreSQL
  • Python
  • Pydantic (a python library)
  • Distributed systems

Job Tags

Full time,

Similar Jobs

UCLA Health

RN, Per Diem - Santa Monica Main OR Job at UCLA Health

DescriptionTake on a rewarding role in a Magnet designated and world-class health organization. The staff RN is a professional caregiver who provides patient care and assumes responsibility and accountability for a group of patients via the nursing process.Under direct... 

Mayo Clinic

Registered Nurse - Mayo 5 Endoscopy Center - RN Job at Mayo Clinic

 ...City Jacksonville State FL Remote NO Department Nursing Why Mayo Clinic Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we... 

EssilorLuxottica Group

Sales Associate Sunglass Outfitters, inside Cabela's Job at EssilorLuxottica Group

Requisition ID:884890 Store # :00C155 Sunglass Hut SGO Position: Seasonal/Temporary Total Rewards: Benefits/Incentive Information At Sunglass Hut, we're always in the sun. Youll find a dynamic space with endless possibilities to grow your career. We are...

Lensa

Scrum Master (Hybrid) Job at Lensa

 ...developing your career, while helping others along the way, come join the Broadridge team. We are seeking an experienced and dynamic Scrum Master to lead the Agile development process for our data platform, which is foundational to our accounting and finance team. You will... 

Johnson Memorial Health Services

RN or LPN, Care Center: 72 hrs/pay period (Night & Wknd) Job at Johnson Memorial Health Services

 ...Job Summary Supervising the day-to-day nursing activities performed by CNAs in accordance...  ...Director of Nursing Services to ensure that the highest degree of quality care is maintained at...  ...or LPN, Care Center Charge Nurse: 72 hrs/pay period (Night & Wkend) position is...