How to page REST calls in a future with Dispatch and Scala -


i use scala , dispatch json paged rest api. reason use futures dispatch here because want execute calls fetchissuesbyfile in parallel, because function result in many rest calls every lookupid (1x findcomponentkeybyrest, n x fetchissuepage, n number of pages yielded rest api).

here's code far:

def fetchissuepage(componentkey: string, pageindex: int): future[json4s.jvalue]  def extractpaginginfo(json: org.json4s.jvalue): paginginfo  def extractissues(json: org.json4s.jvalue): seq[issue]  def findallissuesbyrest(componentkey: string): future[seq[issue]] = {   future {     var paging = paginginfo(pageindex = 0, pages = 0)     val allissues = mutable.listbuffer[issue]()      {       fetchissuepage(componentkey, paging.pageindex) oncomplete {         case success(json) =>           allissues ++= extractissues(json)           paging = extractpaginginfo(json)         case _ => //todo error handling       }     } while (paging.pageindex < paging.pages)      allissues // (1)   } }  def findcomponentkeybyrest(lookupid: string): future[option[string]]  def fetchissuesbyfile(lookupid: string): future[(string, option[seq[issue]])] =   findcomponentkeybyrest(lookupid).flatmap {     case some(key) => findallissuesbyrest(key).map(       issues => // (2) process issues     )     case none => //todo error handling   } 

the actual problem never collected issues findallissuesbyrest (1) (i.e., sequence of issues empty) when try process them @ (2). ideas? also, pagination code isn't functional, i'm open ideas on how improve scala.

any appreciated.

thanks, michael

i think like:

def findallissuesbyrest(componentkey: string): future[seq[issue]] =    // fetch first page   fetchissuepage(componentkey, 0).flatmap { json =>     val nbpages = extractpaginginfo(json).pages // total number of pages     val firstissues = extractissues(json)       // first issues      // other pages     future.sequence((1 nbpages).map(page => fetchissuepage(componentkey, page)))       // issues other pages       .map(pages => pages.map(extractissues))       // combine first issues other issues       .flatmap(issues => (firstissues +: issues).flatten)   } 

Comments

Popular posts from this blog

twig - Using Twigbridge in a Laravel 5.1 Package -

jdbc - Not able to establish database connection in eclipse -

firemonkey - How do I make a beep sound in Android using Delphi and the API? -