Wrapping Access to Web-Services in R-functions.

One of the great features of R is the possibility to quickly access web-services. While some companies have the habit and policy to document their APIs, there is still a large chunk of undocumented but great web-services that help the regular data scientist.

In the following short post, I will show how we can turn a simple web-serivce in a nice R-function. The example I am going to use is the linguee translation service: DeepL. Just as google translate, Deepl features a simple text field. When a user types in text, the translation appears in a second textbox. Users can choose between the languages.

In order to see how the service works in the backend, let’s have a quick look at the network traffic. For that we open the browser’s developer tools and jump to the network tab. Next, we type in a sentence and see which requests (XHR) are made. The interface repeatedly sends JSON requests to the following endpoint: “https://www.deepl.com/jsonrpc”.

Looking at a single request we can quickly identify the parameters that we typed in (grey area, in the lower right corner). We copy these in r and assign them to a variable.

str<-'{"jsonrpc":"2.0","method":"LMT_handle_jobs","params":{"jobs":[{"kind":"default","raw_en_sentence":"R is awesome"}],"lang":{"user_preferred_langs":["DE","EN","FR"],"source_lang_user_selected":"auto","target_lang":"FR"},"priority":-1},"id":11}'

Using a service to format the json (e.g. https://jsonformatter.curiousconcept.com/) we can turn the blob in a well readable json file. Next, we convert the JSON string in a R object (a nested list) by using a simple JSON to R language translation:

require(stringr)str1<-str_replace_all(str,":","=")str2<-str_replace_all(str1,"\\[","list(")str3<-str_replace_all(str2,"\\]",")")str4<-str_replace_all(str3,"\\{","list(")str5<-str_replace_all(str4,"\\}",")")eval(parse(text=str5))

Finally, we evaluate the string as R-code, this gives us the DeepL web-services’ parameters as an R nested list. All we have to do now is wrap the parameters in a R function and use variables to change the important ones:

require(rjson)require(httr)deepLTranslate<-function(text="R is awesome",from_lang="EN",to_lang="DE"){BASE_URL='https://www.deepl.com/jsonrpc'JSONRPC_VERSION='2.0'DEEPL_METHOD='LMT_handle_jobs'params=list('jsonrpc'=JSONRPC_VERSION,'method'=DEEPL_METHOD,params=list('jobs'=list(list('kind'="defaut",'raw_en_sentence'=text)),'lang'=list('user_preferred_langs'=list(from_lang,to_lang),'source_lang_user_selected'=from_lang,'target_lang'=to_lang)))res<-POST(BASE_URL,body=toJSON(params))co<-content(res,"text")if(res$status_code==200){return(fromJSON(co))}else{return(co)}}#### excute the function with defaults ...deepLTranslate()

## $id ## [1] 0 ##  ## $jsonrpc ## [1] "2.0" ##  ## $result ## $result$source_lang ## [1] "EN" ##  ## $result$source_lang_is_confident ## [1] 0 ##  ## $result$target_lang ## [1] "DE" ##  ## $result$translations ## $result$translations[[1]] ## $result$translations[[1]]$beams ## $result$translations[[1]]$beams[[1]] ## $result$translations[[1]]$beams[[1]]$num_symbols ## [1] 5 ##  ## $result$translations[[1]]$beams[[1]]$postprocessed_sentence ## [1] "R ist fantastisch" ##  ## $result$translations[[1]]$beams[[1]]$score ## [1] -5000.6 ##  ## $result$translations[[1]]$beams[[1]]$totalLogProb ## [1] -4.37026 ##  ##  ## $result$translations[[1]]$beams[[2]] ## $result$translations[[1]]$beams[[2]]$num_symbols ## [1] 5 ##  ## $result$translations[[1]]$beams[[2]]$postprocessed_sentence ## [1] "R ist großartig" ##  ## $result$translations[[1]]$beams[[2]]$score ## [1] -5000.64 ##  ## $result$translations[[1]]$beams[[2]]$totalLogProb ## [1] -4.6875 ##  ##  ## $result$translations[[1]]$beams[[3]] ## $result$translations[[1]]$beams[[3]]$num_symbols ## [1] 6 ##  ## $result$translations[[1]]$beams[[3]]$postprocessed_sentence ## [1] "R ist fantastisch." ##  ## $result$translations[[1]]$beams[[3]]$score ## [1] -5000.67 ##  ## $result$translations[[1]]$beams[[3]]$totalLogProb ## [1] -5.57148 ##  ##  ## $result$translations[[1]]$beams[[4]] ## $result$translations[[1]]$beams[[4]]$num_symbols ## [1] 6 ##  ## $result$translations[[1]]$beams[[4]]$postprocessed_sentence ## [1] "R ist großartig." ##  ## $result$translations[[1]]$beams[[4]]$score ## [1] -5000.72 ##  ## $result$translations[[1]]$beams[[4]]$totalLogProb ## [1] -6.03852 ##  ##  ##  ## $result$translations[[1]]$timeAfterPreprocessing ## [1] 0 ##  ## $result$translations[[1]]$timeReceivedFromEndpoint ## [1] 311 ##  ## $result$translations[[1]]$timeSentToEndpoint ## [1] 0 ##  ## $result$translations[[1]]$total_time_endpoint ## [1] 1

I hope the post helps you turn more web-services into R-functions/packages. If you are looking for other translation services have a look at the translate or translateR packages.

var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) {var s = d.createElement(t); s.type = 'text/javascript'; s.async = true;s.src = '//cdn.viglink.com/api/vglnk.js';var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));