Building pipelines with IAsyncEnumerable

This repository demonstrates how to use IAsyncEnumerable and System.Linq.Async to build pipelines in C#.

Important

This repository doesn't cover all the possible concerns such as error handling, cancellation, backpressure, performance, etc. It's just a simple demonstration of how to build pipelines with IAsyncEnumerable and System.Linq.Async.

dotnet example --list

╭─────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
│ Example                                 │ Description                                                                                │
├─────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ CalculateWordCountPipeline              │ Demonstrates how to build async-enumerable pipelines based on standard LINQ operators      │
│ CalculateWordCountFileWatcherPipeline   │ Demonstrates how to combine async-enumerable pipelines with IObservable. E.g: file watcher │
│ CalculateWordCountBatchPipeline         │ Demonstrates how to use batching in async-enumerable pipelines                             │
│ TextSummarizationAndAggregationPipeline │ Demonstrates how to build custom async-enumerable operators                                │
╰─────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────╯

Demo: CalculateWordCountPipeline

demo1.mp4

Demo: CalculateWordCountFileWatcherPipeline

demo2.mp4

Demo: CalculateWordCountBatchPipeline

demo3.mp4

Demo: TextSummarizationAndAggregationPipeline

demo4.mp4

References

• https://learn.microsoft.com/en-us/archive/msdn-magazine/2019/november/csharp-iterating-with-async-enumerables-in-csharp-8
• https://github.com/dotnet/reactive
• https://github.com/microsoft/semantic-kernel/tree/main/dotnet/notebooks