4 months ago
(Big) Data Engineer
• Responsible for the architecture design and R&D of the company's platform, ensuring project schedule and quality.
• Responsible for the verification and implementation of new open source technologies, and promoting technology evolution of the data platform
• Enable rapid development of company's business through data driving, and continuously improve the core competitiveness of the company
• Degree or above in Computer Science, Computer Engineering, Math or a related field with strong experience in java and processing big data.
• Familiar w ith Java or Scala development, understand principle of JVM and familiar with common framew ork (Spring, Mybatis)
• Familiar with distributed system, caching, message and other mechanism with experience in Zookeeper, Redis, Kafka
• Familiar with Hadoop, Spark, Flink, Clickhouse, Hive, ElasticSearch, DataX, Canal and other open source software on big data processing
• Experienced in Python development (pyspark, flask)
• Experienced in AWS EMR big data architecture, and implementation is preferred
• Experienced in various platform (front, middle, end) or microservice design and development, focusing on high availability and scalability of architecture