Ruby Thread Pool Gem

Be an unopinionated toolbox that provides useful utilities without debating which is better or why.

Ruby thread pool gem.

You can find examples of thread pool implementations in gems like puma concurrent ruby celluloid pmap parallel and ruby thread. Inspired by erlang clojure scala haskell f c java and classic concurrency patterns. Instantly publish your gems and then install them use the api to find out more about available gems. A thread pool is an abstraction for re using a limited number of threads to perform concurrent work.

All the implementations i looked at were either buggy or wasted cpu resources for no apparent reason for example used a sleep of 0 01 seconds to then check for readiness and stuff like this. A number of ruby gems exist to support background processing i e saving jobs in a queue and processing them later without blocking the current thread. This allows us to specify a static number of. Become a contributor and improve the site yourself.

In the following example we are using the celluloid gem to do this for you. Create a queue data structure to represent the set of work that needs to be done. Thread pools allow you to control the number of active threads at any given time. Instantly publish your gems and then install them use the api to find out more about available gems.

For this post i ll use sidekiq and redis an open source key value cache and store. Modern concurrency tools for ruby. The design goals of this gem are. You could build your own pool but i wouldn t recommend it.

The connection pool gem will allow us to create a pool of connections which we can use to grab a connection and return it after we re done using it. Gem install thread usage pool. If we happen to be on mri we could probably. A queue is a basic stack like data structure on which we can call push and pop queue is thread safe so if multiple threads access it at the same time it will maintain consistency.

Notable examples include sidekiq resque delayed job and beanstalkd. Gem thread or install it yourself as. We could use a mutex but in most cases a queue will work and is simpler to reason about. Celluloid is now unmaintaned but the general idea of worker pools still applies.