Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/55 link to data source #71

Merged
merged 46 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
aaf2c95
download inputs from data source
arunge Jan 24, 2024
373ce95
remove .Renviron
arunge Jan 24, 2024
e90f503
radiobuttons for link import
arunge Jan 25, 2024
5ae2f95
add draft to import a link to data
arunge Jan 26, 2024
404ce6f
clean up loadImport
arunge Jan 30, 2024
9a54f17
extract getDataSource
arunge Feb 13, 2024
957075f
use input$source for type
arunge Feb 13, 2024
e52cfd5
use loadImport in dataLinkUpload
arunge Feb 19, 2024
440e083
refactor loadDataWrapper
arunge Feb 19, 2024
5097253
Merge branch 'main' into feat/55_linkToDataSource
arunge Feb 19, 2024
efa17d9
fix namespace and docu
arunge Feb 19, 2024
6de4f28
fix to and from json, hide instead of disable
arunge Feb 20, 2024
98ff3be
download mergeList with links to data, refactor formatColumnNames()
arunge Feb 21, 2024
9fcab9c
add message if no internet
arunge Feb 27, 2024
fa6bc88
fix conditionalPanel
arunge Feb 27, 2024
aa2419e
extract loadFileFromLink
arunge Mar 4, 2024
837e804
pipe withProgress
arunge Mar 4, 2024
3d50e58
fix checks, add loop for import from links
arunge Mar 5, 2024
1e2efbc
add logDebug
arunge Mar 5, 2024
f333ce5
comment regarding loading data from last inputs from link
arunge Mar 5, 2024
4f9d3c2
update news.md
arunge Mar 5, 2024
4e4aaa3
add issue to news.md
arunge Mar 5, 2024
63044e8
prepare loading data link from remote folder
arunge Mar 14, 2024
8ae49d4
exchange mainfolder instead of pathToLocal
arunge Mar 14, 2024
a6ccc03
split file for 'loadImport'
arunge Mar 14, 2024
2a1009d
fix docu
arunge Mar 14, 2024
5bf3d91
rename function
arunge Mar 15, 2024
968178d
split file selectData.R, remove getNrow, split function getDataSource
arunge Mar 15, 2024
39410c4
rename file
arunge Mar 15, 2024
43792ed
de-nesting selectData module
arunge Mar 15, 2024
517a755
optimize parameters for helptext
arunge Mar 15, 2024
aba8070
update docu for param rPackageName
arunge Mar 15, 2024
f3e340d
update structure of dataLink after shifting modules
arunge Mar 15, 2024
aaf9e84
make upload of link work for "remote" links
arunge Mar 15, 2024
6a7f671
fix bracket for model upload
arunge Mar 15, 2024
c931103
remove some params
arunge Mar 15, 2024
ada0408
fix docu and warning
arunge Mar 15, 2024
118734a
download and upload of sql query with data link
arunge Mar 18, 2024
fcbce25
merge updates from main
arunge Mar 18, 2024
05547f6
update/add logDebug
arunge Mar 19, 2024
94b4864
reset remote after openPopUp
arunge Mar 19, 2024
ddd7943
fix button labels, hide file specs for remote link
arunge Mar 20, 2024
bfe9426
add/update helpText
arunge Mar 20, 2024
451c3ed
rename folder
arunge Mar 20, 2024
2825ec2
update docuk and placeholder
arunge Mar 20, 2024
97bed1a
update news.md
arunge Mar 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .Renviron

This file was deleted.

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
.Rhistory
.RData
.Ruserdata
.Renviron
docs
5 changes: 3 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: DataTools
Type: Package
Title: Data Tools for Shiny Apps
Version: 24.01.0
Version: 24.03.0
Authors@R: c(
person("Antonia", "Runge", email = "antonia.runge@inwt-statistics.de", role = c("cre", "aut"))
)
Expand All @@ -17,6 +17,7 @@ Imports:
htmltools,
httr,
futile.logger,
jsonlite,
magrittr,
openxlsx,
readxl,
Expand All @@ -37,5 +38,5 @@ Suggests:
devtools,
lintr (>= 1.0.2),
testthat (>= 2.0.0)
RoxygenNote: 7.2.3
RoxygenNote: 7.3.1
URL: https://pandora-isomemo.github.io/data-tools/
4 changes: 4 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ export(startApplication)
export(tryCatchWithWarningsAndErrors)
export(updateListNamesIfDuplicate)
export(updateNameEntryIfDuplicate)
export(updateUserInputs)
export(uploadModelServer)
export(uploadModelUI)
import(shiny, except = c(renderDataTable, dataTableOutput))
Expand All @@ -40,6 +41,7 @@ importFrom(RSQLite,SQLite)
importFrom(RSQLite,dbConnect)
importFrom(RSQLite,dbGetQuery)
importFrom(RSQLite,dbListTables)
importFrom(RSQLite,dbRemoveTable)
importFrom(RSQLite,dbWriteTable)
importFrom(dplyr,full_join)
importFrom(dplyr,inner_join)
Expand All @@ -49,6 +51,8 @@ importFrom(htmltools,save_html)
importFrom(httr,GET)
importFrom(httr,content)
importFrom(httr,timeout)
importFrom(jsonlite,read_json)
importFrom(jsonlite,write_json)
importFrom(magrittr,"%>%")
importFrom(openxlsx,loadWorkbook)
importFrom(readxl,excel_sheets)
Expand Down
7 changes: 7 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# DataTools

## Version 24.03.0

- option to download 'import links' to the imported data if data loading was successful in advance (#55)
- stores only user inputs to file imports, not the data itself
- all links to data that were 'submitted for data preparation' are stored also
- option to upload 'import links' containing user inputs for data imports from a local .json file (#55)

## Version 24.01.0

Integration of rgpt3 functions into data tools package
Expand Down
3 changes: 2 additions & 1 deletion R/00-NAMESPACE.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
#' @importFrom DT datatable renderDataTable dataTableOutput
#' @importFrom htmltools save_html
#' @importFrom httr GET content timeout
#' @importFrom jsonlite read_json write_json
#' @importFrom openxlsx loadWorkbook
#' @importFrom Pandora callAPI getFileTypes getNetworks getRepositories getResources loadData
#' formatRepositoryList
#' @importFrom readxl excel_sheets
#' @importFrom RSQLite dbConnect dbGetQuery dbListTables dbWriteTable SQLite
#' @importFrom RSQLite dbConnect dbGetQuery dbListTables dbWriteTable dbRemoveTable SQLite
#' @importFrom shinyAce aceEditor updateAceEditor
#' @importFrom shinyalert shinyalert
#' @importFrom shinyjs alert disable disabled enable hide hidden info reset runjs show useShinyjs
Expand Down
9 changes: 5 additions & 4 deletions R/01-toolsPanel.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ toolsImportUI <- function(id) {
sidebarPanel(
width = 2,
style = "position:fixed; width:15%; max-width:350px; overflow-y:auto; height:88%",
importDataUI(ns("localData"), "Import Data"),
importDataUI(ns("localData"), "Import Data (local)"),
tags$br(),
tags$br(),
importDataUI(ns("ckanData"), "Import CKAN Data"),
Expand Down Expand Up @@ -51,7 +51,8 @@ toolsImportServer <- function(id) {
customErrorChecks = list(reactive(checkErrorNoNumericColumns)),
ckanFileTypes = config()[["ckanFileTypes"]],
ignoreWarnings = TRUE,
defaultSource = "ckan"
defaultSource = "ckan",
rPackageName = config()[["rPackageName"]]
)

importedBatchData <- importDataServer(
Expand All @@ -62,7 +63,8 @@ toolsImportServer <- function(id) {
ignoreWarnings = TRUE,
defaultSource = config()[["defaultSource"]],
batch = TRUE,
outputAsMatrix = TRUE
outputAsMatrix = TRUE,
rPackageName = config()[["rPackageName"]]
)

importedModel <- importDataServer(
Expand All @@ -73,7 +75,6 @@ toolsImportServer <- function(id) {
ignoreWarnings = TRUE,
defaultSource = config()[["defaultSource"]],
importType = "model",
mainFolder = config()[["mainFolder"]],
fileExtension = config()[["fileExtension"]],
rPackageName = config()[["rPackageName"]]
)
Expand Down
22 changes: 16 additions & 6 deletions R/02-downAndUpload.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@ downUploadButtonUI <-
#' downUploadButton module server
#'
#' @param id module id
#' @param rPackageName (character) name of the package (as in the description file) in which this
#' module is applied, e.g. "mpiBpred"
#' @param title title used inside the modal
#' @inheritParams importOptions
#' @inheritParams downloadModelServer
#' @inheritParams uploadModelUI
#' @inheritParams uploadModelServer
Expand Down Expand Up @@ -147,8 +146,6 @@ downloadModelUI <- function(id, title = NULL, label = "Download", width = NULL)
#' @param dat (reactive) user data
#' @param inputs (reactiveValues) reactiveValues list of user inputs, in most cases just the "inputs" list
#' @param model (reactive) model output object
#' @param rPackageName (character) name of the package (as in the description file) in which this
#' module is applied, e.g. "mpiBpred"
#' @param subFolder (character) (optional) subfolder containing loadable .zip files
#' @param fileExtension (character) (optional) app specific file extension, e.g. "resources",
#' "bpred", "bmsc"
Expand All @@ -162,6 +159,7 @@ downloadModelUI <- function(id, title = NULL, label = "Download", width = NULL)
#' file is a connection.
#' @param compressionLevel A number between 1 and 9. 9 compresses best, but it also takes the
#' longest.
#' @inheritParams importOptions
#'
#' @export
downloadModelServer <-
Expand Down Expand Up @@ -334,8 +332,7 @@ uploadModelUI <- function(id,
#' @param mainFolder (character) folder containing all loadable .zip files. For most apps this
#' is folder "predefinedModels". In most apps it can be found under "inst/app/".
#' @param subFolder (character) (optional) subfolder containing loadable .zip files
#' @param rPackageName (character) If not NULL, than the uploaded file must come from this R
#' package
#' @inheritParams importOptions
#' @inheritParams remoteModelsServer
#'
#' @export
Expand Down Expand Up @@ -422,6 +419,19 @@ getPathToLocal <- function(mainFolder, subFolder, rPackageName = NULL) {
}
}

getSpecsForRemotes <- function(importType) {
if (importType != "data") {
folder <- "predefinedModels"
extension <- "zip"
} else {
folder <- "dataLinks"
extension <- "json"
}

list(folder = folder,
extension = extension)
}

dataLoadedAlert <-
function(warnings,
uploadedVersion,
Expand Down
Loading