Posted on Friday, Dec 2, 2022
In this episode we databases, various types, how they work, etc.

Show Notes

Available on your favorite podcast platform.

In this episode, we are joined by Blake Miner to discuss databases.

Show notes:

  • Introduction
    • Blake’s background, what type of work he does, company information, etc.
    • Data has substantial value
  • What is a database?
    • General overview - store data, process it, and look it up later
    • Important that data stays intact in the event of a app/system crash, power loss, etc.
    • Important that conflicting concurrent changes are handled appropriately
    • What safety guarantees do I get? -> Transactions / ACID - Atomicity, Consistency, Isolation, Durability
      • atomicity -> abortability guarantee for a group of writes (i.e. upon failure)
      • consistency -> guarantee that certain (app-specific; sometimes database-specific) constraints are not violated
      • isolation -> describes how database handles concurrent edits of the same data. Serializable is the strongest guarantee. Snapshot isolation is very common.
      • durability -> storage guarantee (i.e. upon commit, data has fsync‘ed to disk). Study: SSDs get 1 bad block in first 4 years; disaster could wipe out an entire data center; nothing is perfect.
    • Storage / lookup: Data models / data structures
    • Derived data; caching
    • Concurrency; consistency & isolation
  • Reactivity/Realtime
    • tradeoffs between reacting to DB changes vs using a message bus
    • streaming platforms such as NATS Jetstream and Kafka
  • SQL
    • Tables and rows -> array of structs
    • SQL -> Declarative query language (query engine determines implementation details at runtime)
    • B-tree Indexes
  • NoSQL Databases
    • “NoSQL” -> a marketing term
  • Redis
    • Data structure store
    • In-memory vs. disk-based
  • Embedded databases (i.e. API vs. commands over TCP)
  • Designing Data-Intensive Applications by Martin Kleppmann. Blake recommends this book.

Discuss this episode at our community site.


Cliff Brake

Cliff Brake

Cliff has been developing products for a long time. See BEC and Github for more information.

Khem Raj

Khem Raj

Khem is an OpenEmbedded maintainer.