@@ -1,48 +1,24 @@

 Package: RGMQL

 Type: Package

 Title: GenoMetric Query Language for R/Bioconductor

-Version: 0.99.38

+Version: 0.99.40

 Author: Simone Pallotta, Marco Masseroli

 Maintainer: Simone Pallotta <simonepallotta@hotmail.com>

 Description: This package brings the GenoMetric Query Language (GMQL)

     functionalities into the R environment. GMQL is a high-level, declarative

-    language to query and compare multiple and heterogeneous genomic datasets

-    for biomedical knowledge discovery. It allows expressing easily queries and

-    processing over genomic regions and their metadata, to extract genomic regions 

-    of interest and compute their properties. GMQL adopts algorithms designed 

-    for big data and their efficient implementation using cloud-computing 

-    technologies, including Apache Hadoop framework and Spark engine; 

-    these make GMQL able to run on modern high performance computing

-    infrastructures, CPU clusters and network infrastructures, in order to achieve

-    scalability and performance on big data. This RGMQL package is built over 

-    a scalable data management engine written in Scala programming language, 

-    released as Scala API; it provides a set of functions to create, 

-    manipulate and extract genomic data from different data sources both 

-    from local and remote datasets. These RGMQL functions allow performing 

-    complex queries and processing without knowing the GMQL syntax,

-    but leveraging on R idiomatic paradigm and logic. RGMQL provides two different

-    approaches in writing GMQL queries and processing scripts: a) REST calls b)

-    standard R APIs The REST approach let users to log into a remote infrastructure

-    where a GMQL system is installed, and manage remote big genomic datasets hosted

-    in cluster-based repository. User can download an entire remote dataset into

-    local folder, upload local datasets into the remote repository or compiling

-    and running a textual query or processing script just invoking the right RGMQL

-    functions. Multiple REST invocations can be invoked and run concurrently on

-    remote infrastructure allowing user to monitor the progress status of every

-    call. Many other REST functionalities are available in order to allow a complete

-    interaction with remote infrastructure. The R APIs approach lets user work with

-    local or remote datasets using batch-like style where single invocations must

-    be invoked sequentially; with this approach all GMQL queries and processing

-    can be written as a sequence of RGMQL functions. Unlike other similar packages,

-    every RGMQL function simply builds a query, with no intermediate result shown

-    (except for a few functions that execute queries and for some utility functions

-    for interoperability with other packages) The RGMQL package also provides a rich

-    set of ancillary classes that allow sophisticated input/output management and

-    sorting, such as ASC, DESC, BAG, MIN, MAX, SUM, AVG, MEDIAN, STD, Q1, Q2, Q3,

-    and several others; these classes are used only to build predicates and complex

-    conditions taken as input by RGMQL functions; Note that many RGMQL functions are

-    not directly executed in R environment, but are deferred until real execution is

-    issued.

+    language to manage heterogeneous genomic datasets for biomedical purposes, 

+	using simple queries to process genomic regions and their metadata and properties.

+	GMQL adopts algorithms efficiently designed for big data using cloud-computing 

+    technologies (like Apache Hadoop and Spark) allowing GMQL to run on modern

+	infrastructures, in order to achieve scalability and high performance.

+	It allows to to create, manipulate and extract genomic data from different 

+	data sources both locally and remotely. Our RGMQL functions allow complex 

+	queries and processing leveraging on the R idiomatic paradigm. 

+	The RGMQL package also provides a rich set of ancillary classes that allow

+	sophisticated input/output management and sorting, such as:

+	ASC, DESC, BAG, MIN, MAX, SUM, AVG, MEDIAN, STD, Q1, Q2, Q3 (and many others).

+	Note that many RGMQL functions are not directly executed in R environment, 

+	but are deferred until real execution is issued.

 License: Artistic-2.0

 URL: http://www.bioinformatics.deib.polimi.it/genomic_computing/GMQL/

 Encoding: UTF-8

@@ -64,7 +40,7 @@ Imports:

     glue,

     BiocGenerics

 Depends:

-    R(<= 3.4.2), RGMQLlib

+    R(>= 3.4.2), RGMQLlib

 VignetteBuilder: knitr

 Suggests: 

     BiocStyle,

@@ -40,7 +40,7 @@ import_gmql <- function(dataset_path, is_gtf)

 {

     datasetName <- sub("/*[/]$","",datasetName)

     if(basename(datasetName) !="files")

-        datasetName <- paste0(datasetName,"/files")

+        datasetName <- file.path(datasetName,"files")

     if(!dir.exists(datasetName))

         stop("Directory does not exists")

@@ -78,8 +78,8 @@ import_gmql <- function(dataset_path, is_gtf)

 {

     datasetName <- sub("/*[/]$","",datasetName)

     if(basename(datasetName) !="files")

-        datasetName <- paste0(datasetName,"/files")

-

+        datasetName <- file.path(datasetName,"files")

+    

     if(!dir.exists(datasetName))

         stop("Directory does not exists")

@@ -94,7 +94,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

     if(!dir.exists(dir_out))

         dir.create(dir_out)

-    files_sub_dir <- paste0(dir_out,"/files")

+    files_sub_dir <- file.path(dir_out,"files")

     dir.create(files_sub_dir)

     cnt = .counter()

     #col_names <- .get_schema_names(samples)

@@ -102,7 +102,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

     {

         #write region

         lapply(samples,function(x,dir){

-            sample_name = paste0(dir,"/S_",cnt(),".gtf")

+            sample_name <- file.path(dir,paste0("S_",cnt(),".gtf"))

             g <- rtracklayer::export(x,sample_name,format = "gtf")

         },files_sub_dir)

         cnt = .counter(0)

@@ -110,7 +110,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

         #write metadata

         lapply(meta,function(x,dir){

-            sample_name = paste0(dir,"/S_",cnt(),".gtf")

+            sample_name <- file.path(dir,paste0("S_",cnt(),".gtf"))

             .write_metadata(x,sample_name)

         },files_sub_dir)

     }

@@ -118,7 +118,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

     {

         #write region

         lapply(samples,function(x,dir){

-            sample_name = paste0(dir,"/S_",cnt(),".gdm")

+            sample_name <- file.path(dir,paste0("S_",cnt(),".gdm"))

             region_frame <- data.frame(x)

             write.table(region_frame,sample_name,col.names = FALSE,

                             row.names = FALSE, sep = '\t',quote = FALSE)

@@ -129,7 +129,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

         #write metadata

         lapply(meta,function(x,dir){

-            sample_name = paste0(dir,"/S_",cnt(),".gdm")

+            sample_name <- file.path(dir,paste0("S_",cnt(),".gdm"))

             .write_metadata(x,sample_name)

         },files_sub_dir)

     }

@@ -177,7 +177,7 @@ export_gmql <- function(samples, dir_out, is_gtf)

         node_list <- c(fixed_element, columns)

     }

-    schema <- paste0(directory,"/granges.schema")

+    schema <- file.path(directory,"granges.schema")

     root <- xml2::xml_new_root("gmqlSchemaCollection")

     xml2::xml_attr(root,"name") <- "DatasetName_SCHEMAS"

     xml2::xml_attr(root,"xmlns") <- "http://genomic.elet.polimi.it/entities"

@@ -76,8 +76,8 @@ filter_and_extract <- function(data, metadata = NULL,

 {

     datasetName <- sub("/*[/]$","",datasetName)

     if(basename(datasetName) !="files")

-        datasetName <- paste0(datasetName,"/files")

-    

+        datasetName <- file.path(datasetName,"files")

+

     if(!dir.exists(datasetName))

         stop("Directory does not exists")

@@ -165,7 +165,7 @@ gmql_cover <- function(input_data, min_acc, max_acc, groupBy,aggregates,flag)

     }

     else

         metadata_matrix <- .jnull("java/lang/String")

-

+    

     WrappeR <- J("it/polimi/genomics/r/Wrapper")

     response <- switch(flag,

         "COVER" = WrappeR$cover(min_acc, max_acc, join_matrix,

@@ -137,7 +137,7 @@ gmql_materialize <- function(input_data, dir_out, name)

     if(!remote_proc)

     {

         dir_out <- sub("/*[/]$","",dir_out)

-        res_dir_out <- paste0(dir_out,"/",name)

+        res_dir_out <- file.path(dir_out,name)

         if(!dir.exists(res_dir_out))

             dir.create(res_dir_out)

     }

@@ -81,7 +81,7 @@ read_GMQL <- function(dataset, parser = "CustomParser", is_local = TRUE,

         dataset <- sub("/*[/]$","",dataset)

         if(basename(dataset) !="files")

-            dataset <- paste0(dataset,"/files")

+            dataset <- file.path(dataset,"files")

         schema_XML <- list.files(dataset, pattern = "*.schema$",

                                     full.names = TRUE)

@@ -791,7 +791,7 @@ upload_dataset <- function(url, datasetName, folderPath, schemaName = NULL,

                                     isGMQL = TRUE)

 {

     if(isGMQL)

-        folderPath <- paste0(folderPath,"/files")

+        folderPath <- file.path(folderPath,"files")

     files <- list.files(folderPath,full.names = TRUE)

     if(!length(files))

@@ -961,8 +961,8 @@ download_dataset <- function(url, datasetName, path = getwd())

         print(content)

     else

     {

-        zip_path <- paste0(path,"/",datasetName,".zip")

-        dir_out <- paste0(path,"/")

+        zip_path <- file.path(path,paste0(datasetName,".zip"))

+        dir_out <- file.path(path,"")

         writeBin(content, zip_path)

         unzip(zip_path,exdir = dir_out)

         print("Download Complete")

@@ -63,9 +63,10 @@ This package is built over a GMQL scalable data management engine

 written in Scala programming language, released as Scala API [@githubrepo] 

 providing a set of functions to combine, manipulate, compare, and extract 

 genomic data from different data sources both from local and remote datasets.

-These functions, built extending functionalities available in the R/Bioconductor 

-framework, allow performing complex GMQL processing and queries without 

-knowledge of GMQL syntax, but leveraging on R idiomatic paradigm and logic.

+These functions, built extending functionalities available in the 

+R/Bioconductor framework, allow performing complex GMQL processing and queries 

+without knowledge of GMQL syntax, but leveraging on R idiomatic paradigm 

+and logic.

 # Genomic Data Model

@@ -140,7 +141,8 @@ types of files:

 1. genomic region tab-delimited text files with extension .gdm, or .gtf 

 if in standard GTF format

 2. metadata attribute-value tab-delimited text files with the same fullname 

-(name and extension) of the correspondent genomic region file and extension .meta

+(name and extension) of the correspondent genomic region file and extension 

+.meta

 3. schema XML file containing region attribute names and types

 All these files reside in a unique folder called $files$.

@@ -257,17 +259,17 @@ additional input parameter.

 2. GRangesList:\newline

 For better integration in the R environment and with other R packages, 

-we provide the *read_GRangesList()* function to read directly from R memory/environment

+we provide the *read_GRangesList()* function to read directly from R memory

 using GRangesList as input.

 ```{r, read GRangesList}

 library("GenomicRanges")

 gr1 <- GRanges(seqnames = "chr2",

-	ranges = IRanges(103, 106), strand = "+", score = 5L, GC = 0.45)

+    ranges = IRanges(103, 106), strand = "+", score = 5L, GC = 0.45)

 gr2 <- GRanges(seqnames = c("chr1", "chr1"),

-	ranges = IRanges(c(107, 113), width = 3), strand = c("+", "-"),

-	score = 3:4, GC = c(0.3, 0.5))

+    ranges = IRanges(c(107, 113), width = 3), strand = c("+", "-"),

+    score = 3:4, GC = c(0.3, 0.5))

 grl <- GRangesList("txA" = gr1, "txB" = gr2)

 data_out <- read_GRangesList(grl)

@@ -372,10 +374,10 @@ each sample.

 NOTE: GRangesList are contained in the R environment and are not saved on disk.

 With the *rows* parameter it is possible to specify how many rows, for each 

-sample inside the input dataset, are extracted; by default, the *rows* parameter 

-value is $0$, that means all rows are extracted.

-Note that, since we are working with big data, to extract all rows could be very 

-time and space consuming.

+sample inside the input dataset, are extracted; by default, the *rows* 

+parameter value is $0$, that means all rows are extracted.

+Note that, since we are working with big data, to extract all rows could be 

+very time and space consuming.

 ## Remote Processing

@@ -463,9 +465,10 @@ logout_gmql(test_url)

 ### Batch Execution

 This execution type is similar to local processing (syntax, functions, and 

-so on ...) except that materialized data are stored only on the remote repository, 

-from where they can be downloaded locally and imported in GRangesList 

-using the functions in this RGMQL package [(see Import/Export)](#utilities).

+so on ...) except that materialized data are stored only on the remote 

+repository, from where they can be downloaded locally and imported in 

+GRangesList using the functions in this RGMQL package 

+[(see Import/Export)](#utilities).

 Before starting with an example, note that we have to log into remote 

 infrastructure with login function:

@@ -533,7 +536,8 @@ The processing flavour can be switched using the function:

 ```{r, switch mode}

 remote_processing(TRUE)

 ```

-An user can switch processing mode until the first *collect()* has been performed.

+An user can switch processing mode until the first *collect()* has been 

+performed.

 This kind of processing comes from the fact that the *read_GMQL()* function can 

 accept either a local dataset or a remote repository dataset, 

@@ -592,23 +596,25 @@ collect(exon_res)

 execute()

 ```

-As we can see, the two *read_GMQL()* functions above read from different sources: 

-*mut_ds* from local dataset, *HG19_bed_ann* from remote repository.

+As we can see, the two *read_GMQL()* functions above read from different 

+sources: *mut_ds* from local dataset, *HG19_bed_ann* from remote repository.

 If we set remote processing to false (*remote_processing(FALSE)*), 

 the execution is performed locally downloading all needed datasets from remote 

-repositories, otherwise all local datasets are automatically uploaded to the remote 

-GMQL repository associated with the remote system where the processing is performed.

+repositories, otherwise all local datasets are automatically uploaded to the 

+remote GMQL repository associated with the remote system where the processing 

+is performed.

-NOTE: The public datasets cannot be downloaded from a remote GMQL repository by design.

+NOTE: The public datasets cannot be downloaded from a remote GMQL repository 

+by design.

 # Utilities

-The RGMQL package contains functions that allow the user to interface with other 

-packages available in R/Bioconductor repository, e.g., GenomicRanges, and TFARM.

-These functions return GRangesList or GRanges with metadata associated, 

-if present, as data structure suitable to further processing in other R/Bioconductor 

-packages.

+The RGMQL package contains functions that allow the user to interface with 

+other packages available in R/Bioconductor repository, e.g., GenomicRanges, 

+and TFARM. These functions return GRangesList or GRanges with metadata 

+associated, if present, as data structure suitable to further processing 

+in other R/Bioconductor packages.

 ## Import/Export

@@ -655,9 +661,9 @@ matrix

 ```

 *filter_and_extract()* filters the samples in the input dataset based on their 

-specified *metadata*, and then extracts as metadata columns of GRanges the vector 

-of region attributes you specify to retrieve from each filtered sample from the 

-input dataset.

+specified *metadata*, and then extracts as metadata columns of GRanges the 

+vector of region attributes you specify to retrieve from each filtered sample 

+from the input dataset.

 If the $metadata$ argument is NULL, all samples are taken.

 The number of obtained columns is equal to the number of samples left after 

 filtering, multiplied by the number of specified region attributes.