Type Class 101: A practical guide to Monad Transformers (Example)

The last episode of this series covered the motivation behind Monad Transformers and gave some examples of their usage. Now it is time to show a small real world application. By chance I stumpled accross this section of code in an open source project:

private[hajobs] def retriggerJobs(): Future[List[JobStartStatus]] = {
    def retriggerCount: (JobType) => Int = jobManager.retriggerCounts.getOrElse(_, 10)
    jobStatusRepository.getMetadata(limitByJobType = retriggerCount).flatMap { jobStatusMap =>
      val a = jobStatusMap.flatMap { case (jobType, jobStatusList) =>
        triggerIdToRetrigger(jobType, jobStatusList).map { triggerId =>
          logger.info(s"Retriggering job of type $jobType with triggerid $triggerId")
          jobManager.retriggerJob(jobType, triggerId)
        }
      }.toList
      Future.sequence(a)
    }
  }

It does not matter what the code does. We will just hang onto the types to improve it in little steps. Before I do that, we should wonder why we should improve it. For starters, I had a hard time to understand what this thing does. And when I do not understand code, a have a little list of things to look for:

  • there are a couple of flatMaps and maps. Code frequently becomes more readable using for comprehensions.
  • Obviously something is mapped around and then sequenced in the final step. That screams for the use of Future.traverse instead of sequence.
  • Looking at the code in an IDE reveals that some `implicit conversions from scala.Predef happen, in particular conversions of Option to List.

Before we start digging in our crates, I give you a small low down of the types involved:

  • jobStatusRepository.getMetadata(limitByJobType = retriggerCount) returns Future[Map[JobType, List[JobStatus]]]
  • triggerIdToRetrigger(jobType : JobType, jobStatusList : List[JobStatus]) returns Option[UUID]
  • jobManager.retriggerJob(jobType : JobType, triggerId : UUID) returns Future[JobStartStatus]
  • the temporay val a has the type val a: List[Future[JobStartStatus]]
  • The final result is of type Future[List[JobStartStatus]]

The first approach to optimization of readability would be to try to unify the type system a bit and more clearly:

  • The result of jobStatusRepository.getMetadata(...) could be treated as a Future[List[(JobType, List[JobStatus])]]
  • If I had a list of triggerIds I could do Future.traverse(triggerIds)(triggerId => jobManager.retriggerJob(jobType, triggerId))

This yields

private[hajobs] def retriggerJobs(): Future[List[JobStartStatus]] = {
    def retriggerCount: (JobType) => Int = jobType => jobManager.retriggerCounts.getOrElse(jobType, 10)

    for {
      metaDataList <- jobStatusRepository.getMetadata(limitByJobType = retriggerCount).map(_.toList)
      triggerIds <- ???
      jobStartStatusList <- Future.traverse(triggerIds) { triggerId =>
        logger.info(s"Retriggering job of type $jobType with triggerid $triggerId")
        jobManager.retriggerJob(jobType, triggerId)
      }
    } yield {
      jobStartStatusList
    }
  }

Now we have:

  • metaDataList of type List[(JobType, List[JobStatus])]
  • triggerIds must be of type List[UUID]
  • the means to get UUIDs is triggerIdToRetrigger(jobType : JobType, jobStatusList : List[JobStatus])
  • the right handside of the for comprehension (marked as ???) must return a Future[List[UUID]]

So we get:

triggerIds <- Future.successful {
  for {
    (jobType, jobStatusList) <- metaDataList
    triggerId <- triggerIdToRetrigger(jobType, jobStatusList).toList
  } yield triggerId
}

or as a final complete version:

private[hajobs] def retriggerJobs(): Future[List[JobStartStatus]] = {
    def retriggerCount: (JobType) => Int = jobType => jobManager.retriggerCounts.getOrElse(jobType, 10)

    for {
      metaDataList <- jobStatusRepository.getMetadata(limitByJobType = retriggerCount).map(_.toList)
      triggerIds <- successful {
        for {
          (jobType, jobStatusList) <- metaDataList
          triggerId <- triggerIdToRetrigger(jobType, jobStatusList).toList
        } yield triggerId
      }
      jobStartStatusList <- Future.traverse(triggerIds) { triggerId =>
        logger.info(s"Retriggering job of type $jobType with triggerid $triggerId")
        jobManager.retriggerJob(jobType, triggerId)
      }
    } yield {
      jobStartStatusList
    }
  }

To recap - now we have a slightly better version (in my personal opinion) of the code without implicit conversions. We use for comprehensions throughout. On the left hand side we always have Lists, and on the right hand side we always have Future[List[XXX]]. Now if the whole code would not use futures at all, a solution would be very straight forward:

for {
  (_, jobStatusList) <- jobStatusRepository.getMetadata(limitByJobType = retriggerCount).toList
  triggerId <- triggerIdToRetrigger(jobType, jobStatusList).toList
  jobStartStatus <- {
    logger.info(s"Retriggering job of type $jobType with triggerid $triggerId")
    jobManager.retriggerJob(jobType, triggerId)
  }
} yield jobStartStatus

And with this realization we can finally turn to Monad Transformers to the rescue. We choose the ListT[Future, ?] class because we want our Future[List[?]] to behave as if they were lists.

The final result could look like this:

private[hajobs] def retriggerJobs(): Future[List[JobStartStatus]] = {
    def retriggerCount: (JobType) => Int = jobType => jobManager.retriggerCounts.getOrElse(jobType, 10)

    (for {
      (jobType, jobStatusList) <- ListT(jobStatusRepository.getMetadata(limitByJobType = retriggerCount).map(_.toList))
      triggerId                <- ListT(triggerIdToRetrigger(jobType, jobStatusList).toList.point[Future])
      jobStartStatus           <- jobManager.retriggerJob(jobType, triggerId).liftM[ListT]
    } yield jobStartStatus).run
  }

An alternative version which avoids having to lift everything in the ListT monad, is to use flatMapF(A => F[A]) like:

ListT(jobStatusRepository.getMetadata(limitByJobType = retriggerCount).map(_.toList))
      .flatMapF { case (jobType, jobStatusList) => triggerIdToRetrigger(jobType, jobStatusList).toList.point[Future] }
      .flatMapF { triggerId => jobManager.retriggerJob(jobType, triggerId).map(List(_)) }
      .run

I used some helper functions from scalaz, such as liftM. Look them up in the scaladocs, it’s fun. Other helper functions where not used so, to keep the concepts clearer. If you have questions, just ask in the comments.

For further articles in this series: TypeClass101

typeclass101 scalaz scala

Kommentare