Enable `smol` Runtime Support In Resolver Crate

Hey guys! Today, we're diving into a feature request that aims to bring smol runtime support to the resolver crate within the hickory-dns ecosystem. Currently, the resolver crate is heavily reliant on tokio, and the goal is to extend its compatibility to include smol, offering more flexibility for developers. Let's explore the problem, proposed solutions, and alternatives in detail.

The Problem: Limited Runtime Support

Currently, the resolver crate primarily supports the tokio runtime. This limitation poses a challenge for developers who prefer or require the use of the smol runtime in their projects. For those unfamiliar, smol is an asynchronous runtime for Rust that emphasizes lightweightness and efficiency, making it a compelling choice for certain applications.

The core issue is that the existing codebase is tightly coupled with tokio primitives, preventing seamless integration with alternative runtimes like smol. This can be a significant roadblock, especially for projects already invested in the smol ecosystem. The lack of smol support means developers must either refactor their code to accommodate tokio or forego using the resolver crate altogether. Such constraints limit the crate's overall usability and adoption within the broader Rust community.

Furthermore, the decision to remove async-std support in a previous update (#2867) underscores the ongoing need for a more versatile and runtime-agnostic design. While the reasons for removing async-std support were valid (discontinuation), it highlights the importance of finding a sustainable solution that allows the resolver crate to adapt to different runtime environments without incurring excessive maintenance overhead. By addressing this limitation, the resolver crate can cater to a wider audience and remain relevant in the evolving landscape of asynchronous Rust programming. Ensuring compatibility with multiple runtimes ultimately enhances the crate's value and broadens its appeal to a more diverse set of users.

Proposed Solution: Mimic async-std Integration

The most straightforward solution involves mirroring the approach previously used for async-std integration. This entails adapting the existing codebase to accommodate the smol runtime by providing smol-specific implementations of asynchronous operations. The key idea is to leverage the existing architectural patterns and extension points to seamlessly integrate smol without requiring a complete overhaul of the codebase. This approach offers a pragmatic way to introduce smol support while minimizing disruption to the current functionality.

By following the precedent set by the async-std integration, developers can efficiently identify the areas of the code that need modification and apply the necessary adaptations. This might involve creating conditional compilation flags or utilizing trait implementations to provide runtime-specific behavior. The goal is to ensure that the resolver crate can seamlessly switch between tokio and smol based on the user's configuration, without introducing compatibility issues or performance regressions.

However, it's important to note that this approach might inherit some of the limitations or challenges encountered during the async-std integration. Therefore, it's crucial to carefully analyze the previous implementation, identify any potential pitfalls, and address them proactively. While this solution provides a quick and relatively easy way to add smol support, it may not be the most sustainable or scalable approach in the long run. It's essential to consider the trade-offs between short-term gains and long-term maintainability when evaluating this option.

Alternatives Considered: Runtime Agnostic Design

An alternative, more ambitious approach involves refactoring the proto and resolver crates to be truly runtime-agnostic. This would entail decoupling the crates from specific runtime implementations like tokio and smol, and instead relying on a set of abstract interfaces that can be implemented by different runtimes. While this approach requires significantly more effort, it offers the potential for greater flexibility and long-term maintainability.

To achieve runtime agnosticism, the RuntimeProvider trait would need to be extended to handle TCP and TLS connections in a generic manner. This would involve defining a set of abstract methods for establishing and managing connections, without relying on tokio-specific types or functions. Additionally, the Spawn trait would need to be adapted to replace the freestanding tokio::spawn usages in the proto crate, allowing different runtimes to provide their own implementations of task spawning.

The majority of the remaining changes would involve weaving the P: RuntimeProvider trait throughout the rest of the codebase, ensuring that all runtime-specific operations are mediated through this abstraction. This would require careful attention to detail and a thorough understanding of the crate's architecture. While this approach is more complex and time-consuming, it offers the potential for a more robust and adaptable design that can seamlessly accommodate new runtimes in the future. By investing in runtime agnosticism, the hickory-dns project can ensure its long-term viability and relevance in the ever-evolving landscape of asynchronous Rust programming.

Detailed Breakdown of the Runtime Agnostic Approach

Let's delve deeper into the runtime-agnostic approach, examining the key components and steps involved in achieving this ambitious goal. This method focuses on abstracting away runtime-specific details, allowing the proto and resolver crates to operate seamlessly with various asynchronous runtimes, including tokio, smol, and potentially others in the future.

Extending the RuntimeProvider Trait

The RuntimeProvider trait serves as the cornerstone of this approach. Currently, it likely provides basic functionalities, but to support a wider range of runtimes, it needs to be extended to handle TCP and TLS connections. This involves defining a set of abstract methods that allow the crate to establish and manage connections without being tied to a specific runtime's implementation.

For example, the trait could include methods like connect_tcp, connect_tls, accept_tcp, and accept_tls, each responsible for initiating the respective type of connection. These methods would return a generic Stream and Sink type, allowing data to be sent and received over the connection. The actual implementation of these methods would be provided by the runtime-specific implementations of the RuntimeProvider trait.

Adapting the Spawn Trait

The Spawn trait is another critical component that needs to be adapted. Currently, the proto crate likely uses tokio::spawn in several places to spawn new asynchronous tasks. To achieve runtime agnosticism, these usages need to be replaced with a generic spawning mechanism that can be implemented by different runtimes.

This can be achieved by defining a spawn method in the Spawn trait that takes a Future as input and spawns it as a new task. The runtime-specific implementations of the Spawn trait would then provide the actual implementation of this method, using their respective task spawning mechanisms. This allows the proto crate to spawn tasks without being tied to a specific runtime's spawning API.

Weaving the P: RuntimeProvider Throughout the Codebase

The most challenging part of this approach is weaving the P: RuntimeProvider trait throughout the rest of the codebase. This involves identifying all the places where runtime-specific operations are being performed and replacing them with calls to the abstract methods defined in the RuntimeProvider trait.

This requires a deep understanding of the crate's architecture and a careful analysis of the code to identify all the runtime-specific dependencies. It also requires a significant amount of refactoring to replace these dependencies with calls to the RuntimeProvider trait. However, once this is done, the crate will be truly runtime-agnostic, allowing it to be used with any runtime that provides an implementation of the RuntimeProvider trait.

Additional Context and Considerations

The resolver crate previously supported async-std, but this support was removed in #2867 due to the discontinuation of async-std. This highlights the challenges of maintaining compatibility with multiple runtimes and the importance of finding a sustainable solution. The runtime-agnostic approach offers a potential solution to this problem, but it requires a significant investment of time and effort.

When considering whether to implement the simpler smol integration or the more ambitious runtime-agnostic design, it's essential to weigh the trade-offs between short-term gains and long-term maintainability. The smol integration provides a quick and relatively easy way to add smol support, but it may not be the most sustainable or scalable approach in the long run. The runtime-agnostic design, on the other hand, offers the potential for greater flexibility and long-term maintainability, but it requires a significant investment of time and effort.

Ultimately, the decision depends on the specific needs and priorities of the hickory-dns project. If the goal is to quickly add smol support with minimal effort, the smol integration may be the best option. However, if the goal is to create a more robust and adaptable design that can seamlessly accommodate new runtimes in the future, the runtime-agnostic design may be the better choice.

In conclusion, enabling smol runtime support in the resolver crate is a valuable goal that can enhance the crate's usability and adoption. Whether through a straightforward integration approach or a more ambitious runtime-agnostic design, the key is to carefully consider the trade-offs and choose the solution that best aligns with the project's long-term vision.

For more information on asynchronous Rust runtimes, check out the Tokio project website.