Schema Registry

Helps with schema problems in Apache Kafka

First local cache is checked for the message schema. In case of cache miss, schema is pulled from the schema registry. An exception will be thrown in the Schema Registry does not have the schema (which should never happen if you set it up properly)

• Kafka takes bytes as an input and publishes them
• There is no data verification in Kafka
• Changes to data shape causes consumers to break
• We need data to be self describable
• Evolve data without breaking downstream consumers
• Schema registry helps manage changes to schemas and provides a way to validate data

Why Not Kafka Broker as Schema Registry?

• Kafka doesn't parse or even read your data (no CPU Usage)
• Kafka would lose zero copy (copy directly to disk)
• Kafka distributes bytes, would require deserialization
• Would lose performance overall

References

Flashcards

What isn't a feature of the Confluent schema registry?

• Store avro data
• Enforce compatibility rules
• Store schemas
  ?
  Store avro data

A consumer application is using KafkaAvroDeserializer to deserialize Avro messages. What happens if message schema is not present in AvroDeserializer local cache?:: Fetches schema from Schema Registry

Using the Confluent Schema Registry, where are Avro schema stored?:: In the _schemas topic

What client protocol is supported for the schema registry? (select two)

• SASL
• Websocket
• HTTP
• HTTPS
• JDBC
  ?
• HTTP
• HTTPS

I am producing Avro data on my Kafka cluster that is integrated with the Confluent Schema Registry. After a schema change that is incompatible, I know my data will be rejected. Which component will reject the data?:: The Confluent Schema Registry is your safeguard against incompatible schema changes and will be the component that ensures no breaking schema evolution will be possible. Kafka Brokers do not look at your payload and your payload schema, and therefore will not reject data

When using the Confluent Kafka Distribution, where does the schema registry reside?:: As a separate JVM component. Schema registry is a separate application that provides RESTful interface for storing and retrieving Avro schemas.

In Avro, removing a field that does not have a default is a breaking schema evolution

In Java, Avro SpecificRecords classes are:: automatically generated from an Avro Schema + a Maven / Gradle Plugin. SpecificRecord is created from generated record classes

In Avro, removing or adding a field that has a default is a full schema evolution

Which of the following is not an Avro primitive type?

• string
• date
• long
• int
• null
  ?
  date

In Avro, adding an element to an enum without a default is a breaking schema evolution