Listen "97: Avro and Elixir with Dave Lucia"
Episode Synopsis
Dave Lucia released version 2.0 of AvroEx, an Elixir library for working with Avro protocols and schemas. To understand Avro, we need an overview of the whole Kafka-centric flow! He gives us a taste of the Enterprise soup of JVM tools used for populating data lakes, all for the business benefit of data analysis and machine learning. We may work on Elixir systems that are tasked with hooking into this other Enterprise-y world and getting an overview of the pieces and awareness of the tools available can really help out! Dave also shares his goal of creating a new Markdown parser for Elixir and updates us on the next chapter of his professional journey.
Show Notes online - http://podcast.thinkingelixir.com/97 (http://podcast.thinkingelixir.com/97)
Elixir Community News
- https://twitter.com/wojtekmach/status/1516533938977705990 (https://twitter.com/wojtekmach/status/1516533938977705990) – Wojtek Mach shared some additional reasons for using NIFs to work with Rust or Zig that aren't performance related.
- https://twitter.com/atyborska93/status/1516773412848422919 (https://twitter.com/atyborska93/status/1516773412848422919) – Angelika Tyborska shared that the Elixir track on Exercism.io got two new fun practice exercises
- http://exercism.org/tracks/elixir/exercises/killer-sudoku-helper (http://exercism.org/tracks/elixir/exercises/killer-sudoku-helper) – Exercism.io Elixir puzzle - Killer Sudoku Helper
- https://exercism.org/tracks/elixir/exercises/state-of-tic-tac-toe (https://exercism.org/tracks/elixir/exercises/state-of-tic-tac-toe) – Exercism.io Elixir puzzle - State of Tic Tac Toe
- https://gleam.run/news/v0.21-introducing-the-gleam-language-server/ (https://gleam.run/news/v0.21-introducing-the-gleam-language-server/) – Gleam now has a language server
- https://twitter.com/cigrainger/status/1518896955921154048 (https://twitter.com/cigrainger/status/1518896955921154048) – Nx Explorer project reaches a new milestone and was released to Hex.
- https://hexdocs.pm/explorer/exploring_explorer.html (https://hexdocs.pm/explorer/exploring_explorer.html) – Includes a “Ten Minutes to Explorer” Livebook guide
- https://twitter.com/MozillaHubs/status/1516469058849312768 (https://twitter.com/MozillaHubs/status/1516469058849312768) – Mozilla hiring for a Senior Elixir Engineer to help work on their Hubs Team to help build open social spaces for the web.
- https://hubs.mozilla.com/ (https://hubs.mozilla.com/)
- https://github.com/mozilla/hubs (https://github.com/mozilla/hubs)
- https://github.com/mozilla/reticulum (https://github.com/mozilla/reticulum)
- https://elixir-lang.org/blog/2021/06/02/social-virtual-spaces-with-elixir-at-mozilla (https://elixir-lang.org/blog/2021/06/02/social-virtual-spaces-with-elixir-at-mozilla) – Elixir blog post from a year ago about the Mozilla Hubs Elixir project.
- https://github.com/elixir-makeup/makeup_diff (https://github.com/elixir-makeup/makeup_diff) – Parker Selbert created makeup_diff for displaying diff markup in ExDocs
- https://www.erlang.org/blog/type-based-optimizations-in-the-jit/ (https://www.erlang.org/blog/type-based-optimizations-in-the-jit/) – Blog post on the Erlang blog looked interesting. About OTP 25 type optimizations for the JIT compiler.
- https://codesync.global/conferences/code-beam-sto-2022/#schedule (https://codesync.global/conferences/code-beam-sto-2022/#schedule) – CodeBeam in Stockholm Sweden speaker lineup and schedule are out.
- https://www.empex.co/mtn (https://www.empex.co/mtn) – Empex MTN in Salt Lake City on May 6
- https://www.elixirconf.eu/ (https://www.elixirconf.eu/) – ElixirConf EU in London on June 9-10
- https://elixirconf.com/events (https://elixirconf.com/events) – ElixirConf US in Colorado on August 30-Sep2
Do you have some Elixir news to share? Tell us at @ThinkingElixir (https://twitter.com/ThinkingElixir) or email at [email protected] (mailto:[email protected])
Discussion Resources
- https://davelucia.com/blog/avro-2.0 (https://davelucia.com/blog/avro-2.0)
- https://github.com/beam-community/avro_ex (https://github.com/beam-community/avro_ex)
- https://avro.apache.org/ (https://avro.apache.org/)
- https://kafka.apache.org/ (https://kafka.apache.org/)
- https://podcast.thinkingelixir.com/75 (https://podcast.thinkingelixir.com/75) – Previous discussion
- https://www.bloomberg.com/professional/solution/bloomberg-terminal/ (https://www.bloomberg.com/professional/solution/bloomberg-terminal/)
- https://theoutline.com/ (https://theoutline.com/)
- https://elm-lang.org/ (https://elm-lang.org/)
- https://developers.google.com/protocol-buffers/ (https://developers.google.com/protocol-buffers/)
- https://thrift.apache.org/ (https://thrift.apache.org/)
- https://github.com/klarna/erlavro (https://github.com/klarna/erlavro)
- https://databricks.com/ (https://databricks.com/)
- https://spark.apache.org/ (https://spark.apache.org/)
- https://en.wikipedia.org/wiki/Extract,transform,load (https://en.wikipedia.org/wiki/Extract,_transform,_load)
- https://databricks.com/glossary/what-is-parquet (https://databricks.com/glossary/what-is-parquet)
- https://www.snowflake.com/workloads/data-lake/ (https://www.snowflake.com/workloads/data-lake/)
- https://github.com/beam-community (https://github.com/beam-community)
- https://github.com/doomspork (https://github.com/doomspork)
- https://github.com/surface-ui/surface (https://github.com/surface-ui/surface)
- https://github.com/pragdave/earmark (https://github.com/pragdave/earmark)
- https://github.com/pragdave (https://github.com/pragdave)
- https://github.com/davydog187/marx (https://github.com/davydog187/marx)
- https://www.youtube.com/watch?v=dmBdFh5N1g4 (https://www.youtube.com/watch?v=dmBdFh5N1g4) – Migrating from Kafka to RabbitMQ at SimpleBet - Why and How
- https://twitter.com/davydog187/status/1512393866653011968 (https://twitter.com/davydog187/status/1512393866653011968) – Announced leaving SimpleBet and co-founding something new.
- https://twitter.com/davydog187/status/1512393877575110668 (https://twitter.com/davydog187/status/1512393877575110668) – Co-Founding Bitfo as the CTO.
- https://www.bitfo.com/ (https://www.bitfo.com/)
- https://ethereumprice.org/ (https://ethereumprice.org/)
- https://www.bitcoinprice.com/ (https://www.bitcoinprice.com/)
- https://defirate.com/ (https://defirate.com/)
Guest Information
- https://twitter.com/davydog187 (https://twitter.com/davydog187) – on Twitter
- https://github.com/davydog187/ (https://github.com/davydog187/) – on Github
- https://davelucia.com (https://davelucia.com) – Blog
Find us online
- Message the show - @ThinkingElixir (https://twitter.com/ThinkingElixir)
- Email the show - [email protected] (mailto:[email protected])
- Mark Ericksen - @brainlid (https://twitter.com/brainlid)
- David Bernheisel - @bernheisel (https://twitter.com/bernheisel)
- Cade Ward - @cadebward (https://twitter.com/cadebward)
Show Notes online - http://podcast.thinkingelixir.com/97 (http://podcast.thinkingelixir.com/97)
Elixir Community News
- https://twitter.com/wojtekmach/status/1516533938977705990 (https://twitter.com/wojtekmach/status/1516533938977705990) – Wojtek Mach shared some additional reasons for using NIFs to work with Rust or Zig that aren't performance related.
- https://twitter.com/atyborska93/status/1516773412848422919 (https://twitter.com/atyborska93/status/1516773412848422919) – Angelika Tyborska shared that the Elixir track on Exercism.io got two new fun practice exercises
- http://exercism.org/tracks/elixir/exercises/killer-sudoku-helper (http://exercism.org/tracks/elixir/exercises/killer-sudoku-helper) – Exercism.io Elixir puzzle - Killer Sudoku Helper
- https://exercism.org/tracks/elixir/exercises/state-of-tic-tac-toe (https://exercism.org/tracks/elixir/exercises/state-of-tic-tac-toe) – Exercism.io Elixir puzzle - State of Tic Tac Toe
- https://gleam.run/news/v0.21-introducing-the-gleam-language-server/ (https://gleam.run/news/v0.21-introducing-the-gleam-language-server/) – Gleam now has a language server
- https://twitter.com/cigrainger/status/1518896955921154048 (https://twitter.com/cigrainger/status/1518896955921154048) – Nx Explorer project reaches a new milestone and was released to Hex.
- https://hexdocs.pm/explorer/exploring_explorer.html (https://hexdocs.pm/explorer/exploring_explorer.html) – Includes a “Ten Minutes to Explorer” Livebook guide
- https://twitter.com/MozillaHubs/status/1516469058849312768 (https://twitter.com/MozillaHubs/status/1516469058849312768) – Mozilla hiring for a Senior Elixir Engineer to help work on their Hubs Team to help build open social spaces for the web.
- https://hubs.mozilla.com/ (https://hubs.mozilla.com/)
- https://github.com/mozilla/hubs (https://github.com/mozilla/hubs)
- https://github.com/mozilla/reticulum (https://github.com/mozilla/reticulum)
- https://elixir-lang.org/blog/2021/06/02/social-virtual-spaces-with-elixir-at-mozilla (https://elixir-lang.org/blog/2021/06/02/social-virtual-spaces-with-elixir-at-mozilla) – Elixir blog post from a year ago about the Mozilla Hubs Elixir project.
- https://github.com/elixir-makeup/makeup_diff (https://github.com/elixir-makeup/makeup_diff) – Parker Selbert created makeup_diff for displaying diff markup in ExDocs
- https://www.erlang.org/blog/type-based-optimizations-in-the-jit/ (https://www.erlang.org/blog/type-based-optimizations-in-the-jit/) – Blog post on the Erlang blog looked interesting. About OTP 25 type optimizations for the JIT compiler.
- https://codesync.global/conferences/code-beam-sto-2022/#schedule (https://codesync.global/conferences/code-beam-sto-2022/#schedule) – CodeBeam in Stockholm Sweden speaker lineup and schedule are out.
- https://www.empex.co/mtn (https://www.empex.co/mtn) – Empex MTN in Salt Lake City on May 6
- https://www.elixirconf.eu/ (https://www.elixirconf.eu/) – ElixirConf EU in London on June 9-10
- https://elixirconf.com/events (https://elixirconf.com/events) – ElixirConf US in Colorado on August 30-Sep2
Do you have some Elixir news to share? Tell us at @ThinkingElixir (https://twitter.com/ThinkingElixir) or email at [email protected] (mailto:[email protected])
Discussion Resources
- https://davelucia.com/blog/avro-2.0 (https://davelucia.com/blog/avro-2.0)
- https://github.com/beam-community/avro_ex (https://github.com/beam-community/avro_ex)
- https://avro.apache.org/ (https://avro.apache.org/)
- https://kafka.apache.org/ (https://kafka.apache.org/)
- https://podcast.thinkingelixir.com/75 (https://podcast.thinkingelixir.com/75) – Previous discussion
- https://www.bloomberg.com/professional/solution/bloomberg-terminal/ (https://www.bloomberg.com/professional/solution/bloomberg-terminal/)
- https://theoutline.com/ (https://theoutline.com/)
- https://elm-lang.org/ (https://elm-lang.org/)
- https://developers.google.com/protocol-buffers/ (https://developers.google.com/protocol-buffers/)
- https://thrift.apache.org/ (https://thrift.apache.org/)
- https://github.com/klarna/erlavro (https://github.com/klarna/erlavro)
- https://databricks.com/ (https://databricks.com/)
- https://spark.apache.org/ (https://spark.apache.org/)
- https://en.wikipedia.org/wiki/Extract,transform,load (https://en.wikipedia.org/wiki/Extract,_transform,_load)
- https://databricks.com/glossary/what-is-parquet (https://databricks.com/glossary/what-is-parquet)
- https://www.snowflake.com/workloads/data-lake/ (https://www.snowflake.com/workloads/data-lake/)
- https://github.com/beam-community (https://github.com/beam-community)
- https://github.com/doomspork (https://github.com/doomspork)
- https://github.com/surface-ui/surface (https://github.com/surface-ui/surface)
- https://github.com/pragdave/earmark (https://github.com/pragdave/earmark)
- https://github.com/pragdave (https://github.com/pragdave)
- https://github.com/davydog187/marx (https://github.com/davydog187/marx)
- https://www.youtube.com/watch?v=dmBdFh5N1g4 (https://www.youtube.com/watch?v=dmBdFh5N1g4) – Migrating from Kafka to RabbitMQ at SimpleBet - Why and How
- https://twitter.com/davydog187/status/1512393866653011968 (https://twitter.com/davydog187/status/1512393866653011968) – Announced leaving SimpleBet and co-founding something new.
- https://twitter.com/davydog187/status/1512393877575110668 (https://twitter.com/davydog187/status/1512393877575110668) – Co-Founding Bitfo as the CTO.
- https://www.bitfo.com/ (https://www.bitfo.com/)
- https://ethereumprice.org/ (https://ethereumprice.org/)
- https://www.bitcoinprice.com/ (https://www.bitcoinprice.com/)
- https://defirate.com/ (https://defirate.com/)
Guest Information
- https://twitter.com/davydog187 (https://twitter.com/davydog187) – on Twitter
- https://github.com/davydog187/ (https://github.com/davydog187/) – on Github
- https://davelucia.com (https://davelucia.com) – Blog
Find us online
- Message the show - @ThinkingElixir (https://twitter.com/ThinkingElixir)
- Email the show - [email protected] (mailto:[email protected])
- Mark Ericksen - @brainlid (https://twitter.com/brainlid)
- David Bernheisel - @bernheisel (https://twitter.com/bernheisel)
- Cade Ward - @cadebward (https://twitter.com/cadebward)
More episodes of the podcast Thinking Elixir Podcast
283: Erlang Turns 27 and React at Risk
16/12/2025
282: Type Systems and View Transitions
09/12/2025
281: Planning for the Unexpected
02/12/2025
280: Dark Matter Developers
25/11/2025
279: Hot Code Upgrades and Hotter AI Takes
18/11/2025
278: WAL-ing Through Database Changes
11/11/2025
277: Searching Across the Hexiverse
04/11/2025
276: Elixir v1.19 Types and Speed
28/10/2025
275: From Slop to Success?
21/10/2025
274: Protocols, Permissions, and Performance
14/10/2025
ZARZA We are Zarza, the prestigious firm behind major projects in information technology.