| Title: | Traceability Engine for Clinical Submission Readiness |
|---|---|
| Description: | Quantifies and explains end-to-end traceability between clinical submission artifacts (ADaM (Analysis Data Model) outputs, derivations, SDTM (Study Data Tabulation Model) sources, specs, code). Builds trace models from metadata and mapping sheets, computes trace levels, and emits standardized R4SUB (Ready for Submission) evidence table rows via 'r4subcore'. |
| Authors: | Pawan Rama Mali [aut, cre, cph] (ORCID: <https://orcid.org/0000-0001-7864-5819>) |
| Maintainer: | Pawan Rama Mali <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.3.0 |
| Built: | 2026-10-11 10:37:22 UTC |
| Source: | https://github.com/r4sub/r4subtrace |
Constructs a directed trace model (nodes + edges + diagnostics) from ADaM metadata, SDTM metadata, and an optional mapping sheet.
build_trace_model( adam_meta, sdtm_meta, mapping = NULL, spec = NULL, config = trace_config_default() )build_trace_model( adam_meta, sdtm_meta, mapping = NULL, spec = NULL, config = trace_config_default() )
adam_meta |
A data.frame of ADaM variable metadata. Must contain
|
sdtm_meta |
A data.frame of SDTM variable metadata. Must contain
|
mapping |
An optional data.frame describing ADaM-to-SDTM mappings.
Must contain |
spec |
Reserved for future use (ADaM spec ingestion). |
config |
A |
A list of class "trace_model" with elements:
nodes: tibble of asset nodes (datasets and variables)
edges: tibble of relationships between nodes
diagnostics: list of tibbles (orphans, ambiguities, conflicts)
config: the configuration used
adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) tm$nodes tm$edgesadam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) tm$nodes tm$edges
Assigns a traceability level (L0–L3) to each ADaM variable in the trace model based on available mapping, derivation text, and confidence scores.
compute_trace_levels(trace_model)compute_trace_levels(trace_model)
trace_model |
A |
Trace levels:
L0: No mapping and no derivation text.
L1: Derivation text present but no SDTM mapping.
L2: Mapping to SDTM variable/domain exists.
L3: Mapping exists AND (confidence >= threshold OR derivation text present alongside mapping).
A tibble with columns: adam_dataset, adam_var, trace_level,
has_mapping, has_derivation_text, n_candidates, max_confidence.
adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGE", "AGEGR1"), label = c("Study ID", "Unique Subject ID", "Age", "Age Group") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID", "AGE"), confidence = c(1.0, 1.0, 0.9) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) compute_trace_levels(tm)adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGE", "AGEGR1"), label = c("Study ID", "Unique Subject ID", "Age", "Age Group") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "USUBJID", "AGE"), label = c("Study ID", "Unique Subject ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID", "AGE"), confidence = c(1.0, 1.0, 0.9) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) compute_trace_levels(tm)
Produces the coverage table an ADRG appendix needs: per-dataset traceability percentages, the untraced variables, and an overall summary row. The report can be returned as a data.frame or written to CSV or Excel for inclusion in a submission package.
export_traceability_report( trace_model, format = c("data.frame", "csv", "xlsx"), path = NULL, traced_min_level = 2L )export_traceability_report( trace_model, format = c("data.frame", "csv", "xlsx"), path = NULL, traced_min_level = 2L )
trace_model |
A |
format |
One of |
path |
Output file path, required for |
traced_min_level |
The minimum trace level that counts as traced.
Passed to |
For "data.frame", the report tibble. For "csv" and "xlsx", the
report is written to path and the path is returned invisibly.
adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGEGR1"), label = c("Study ID", NA) ) sdtm_meta <- data.frame(dataset = "DM", variable = "STUDYID") map <- data.frame( adam_dataset = "ADSL", adam_var = "STUDYID", sdtm_domain = "DM", sdtm_var = "STUDYID", confidence = 1 ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) export_traceability_report(tm)adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGEGR1"), label = c("Study ID", NA) ) sdtm_meta <- data.frame(dataset = "DM", variable = "STUDYID") map <- data.frame( adam_dataset = "ADSL", adam_var = "STUDYID", sdtm_domain = "DM", sdtm_var = "STUDYID", confidence = 1 ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) export_traceability_report(tm)
Draws the derivation graph behind trace_chains() using igraph: ADaM and
SDTM variables as nodes, derivation links as directed edges.
plot_trace_graph(trace_model, adam_derivations = NULL, ...)plot_trace_graph(trace_model, adam_derivations = NULL, ...)
trace_model |
A |
adam_derivations |
Optional ADaM-to-ADaM derivations, as in
|
... |
Passed to |
The igraph object, invisibly.
## Not run: plot_trace_graph(tm, adam_derivations = deriv) ## End(Not run)## Not run: plot_trace_graph(tm, adam_derivations = deriv) ## End(Not run)
Print Trace Model
## S3 method for class 'trace_model' print(x, ...)## S3 method for class 'trace_model' print(x, ...)
x |
A |
... |
Ignored. |
Traces each ADaM variable back to its ultimate source through chains of any
depth, not just a single ADaM-to-SDTM hop. Real submissions often derive an
ADaM variable from another ADaM variable that is itself derived from SDTM
(for example an ADAE variable built from ADSL, which in turn comes from
DM). Shallow tracing misses those links and understates coverage.
trace_chains(trace_model, adam_derivations = NULL, trace_depth = 3L)trace_chains(trace_model, adam_derivations = NULL, trace_depth = 3L)
trace_model |
A |
adam_derivations |
Optional data.frame of ADaM-to-ADaM derivations with
columns |
trace_depth |
Maximum number of hops to follow, a positive integer.
Default |
The ADaM-to-SDTM links come from the trace model's edges. ADaM-to-ADaM links
are supplied through adam_derivations, a data.frame with columns
from_dataset, from_var, to_dataset, to_var, where from is the
derived variable and to is its source. Each ADaM variable is walked toward
its sources until the path reaches an SDTM variable (grounded), dead-ends at
an ADaM variable with no further source (ungrounded), revisits a node
(a cycle), or reaches trace_depth hops.
A tibble with one row per ADaM variable: adam_dataset, adam_var,
max_depth (longest chain in hops), grounded (a path reaches SDTM),
terminal_source, has_cycle, truncated (a path hit trace_depth),
n_paths, and longest_chain (a readable path string).
adam <- data.frame( dataset = c("ADSL", "ADAE"), variable = c("AGE", "AAGE"), label = c("Age", "Analysis Age") ) sdtm <- data.frame(dataset = "DM", variable = "AGE", label = "Age") map <- data.frame( adam_dataset = "ADSL", adam_var = "AGE", sdtm_domain = "DM", sdtm_var = "AGE", confidence = 1 ) tm <- build_trace_model(adam, sdtm, mapping = map) deriv <- data.frame( from_dataset = "ADAE", from_var = "AAGE", to_dataset = "ADSL", to_var = "AGE" ) trace_chains(tm, adam_derivations = deriv)adam <- data.frame( dataset = c("ADSL", "ADAE"), variable = c("AGE", "AAGE"), label = c("Age", "Analysis Age") ) sdtm <- data.frame(dataset = "DM", variable = "AGE", label = "Age") map <- data.frame( adam_dataset = "ADSL", adam_var = "AGE", sdtm_domain = "DM", sdtm_var = "AGE", confidence = 1 ) tm <- build_trace_model(adam, sdtm, mapping = map) deriv <- data.frame( from_dataset = "ADAE", from_var = "AAGE", to_dataset = "ADSL", to_var = "AGE" ) trace_chains(tm, adam_derivations = deriv)
Returns a list of default configuration values for trace model building and evidence emission.
trace_config_default( severity_by_level = c(L0 = "high", L1 = "medium", L2 = "low", L3 = "info"), result_by_level = c(L0 = "fail", L1 = "warn", L2 = "warn", L3 = "pass"), confidence_threshold_L3 = 0.8, uppercase_datasets = TRUE )trace_config_default( severity_by_level = c(L0 = "high", L1 = "medium", L2 = "low", L3 = "info"), result_by_level = c(L0 = "fail", L1 = "warn", L2 = "warn", L3 = "pass"), confidence_threshold_L3 = 0.8, uppercase_datasets = TRUE )
severity_by_level |
Named character vector mapping trace levels to severity. |
result_by_level |
Named character vector mapping trace levels to result. |
confidence_threshold_L3 |
Numeric threshold for L3 classification. A mapping must have confidence >= this value to qualify for L3. |
uppercase_datasets |
Logical; if |
A list of class "trace_config" with elements:
severity_by_level, result_by_level, confidence_threshold_L3,
uppercase_datasets.
cfg <- trace_config_default() cfg$severity_by_level # Override a single setting cfg2 <- trace_config_default(confidence_threshold_L3 = 0.9)cfg <- trace_config_default() cfg$severity_by_level # Override a single setting cfg2 <- trace_config_default(confidence_threshold_L3 = 0.9)
Answers the question a data team asks before touching a source variable: if
this changes, what downstream ADaM variables are affected? Given a trace
model and a variable that is about to change, trace_impact() follows the
derivation graph in reverse and returns every variable that derives from it,
directly or through a chain.
trace_impact(trace_model, changed, adam_derivations = NULL, max_depth = Inf)trace_impact(trace_model, changed, adam_derivations = NULL, max_depth = Inf)
trace_model |
A |
changed |
A single variable identifier as |
adam_derivations |
Optional data.frame of ADaM-to-ADaM derivations with
columns |
max_depth |
Optional maximum number of hops to report. |
The trace model records derivations as edges from a derived variable to its
source. Impact analysis walks those edges backwards from the changed variable
to find its dependents. ADaM-to-ADaM derivations are included when supplied
through adam_derivations, exactly as in trace_chains(), so multi-step
impact (an ADTTE variable built from an ADSL variable built from DM) is
captured. Each dependent is reported once, at its shortest distance from the
change, with a readable path back to it. Circular references are handled and
do not loop.
A tibble with one row per impacted variable: dataset, variable,
role, depth (hops from the change), and path (a readable derivation
path from the impacted variable back to the changed one). Rows are ordered
nearest first. Zero rows means nothing derives from the variable. The
result carries a "changed" attribute with the resolved identifier.
adam <- data.frame( dataset = c("ADSL", "ADAE"), variable = c("AGE", "AAGE"), label = c("Age", "Analysis Age") ) sdtm <- data.frame(dataset = "DM", variable = "AGE", label = "Age") map <- data.frame( adam_dataset = "ADSL", adam_var = "AGE", sdtm_domain = "DM", sdtm_var = "AGE", confidence = 1 ) tm <- build_trace_model(adam, sdtm, mapping = map) deriv <- data.frame( from_dataset = "ADAE", from_var = "AAGE", to_dataset = "ADSL", to_var = "AGE" ) # If DM.AGE changes, what is affected? trace_impact(tm, "DM.AGE", adam_derivations = deriv)adam <- data.frame( dataset = c("ADSL", "ADAE"), variable = c("AGE", "AAGE"), label = c("Age", "Analysis Age") ) sdtm <- data.frame(dataset = "DM", variable = "AGE", label = "Age") map <- data.frame( adam_dataset = "ADSL", adam_var = "AGE", sdtm_domain = "DM", sdtm_var = "AGE", confidence = 1 ) tm <- build_trace_model(adam, sdtm, mapping = map) deriv <- data.frame( from_dataset = "ADAE", from_var = "AAGE", to_dataset = "ADSL", to_var = "AGE" ) # If DM.AGE changes, what is affected? trace_impact(tm, "DM.AGE", adam_derivations = deriv)
Computes summary metrics from evidence rows generated by
trace_model_to_evidence(). Returns key traceability indicators.
trace_indicator_scores(evidence)trace_indicator_scores(evidence)
evidence |
A data.frame of evidence rows (must contain |
A tibble with columns: indicator, value, description.
library(r4subcore) ctx <- r4sub_run_context(study_id = "TEST001", environment = "DEV") adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGE", "AGEGR1"), label = c("Study ID", "Age", "Age Group") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) ev <- trace_model_to_evidence(tm, ctx = ctx) trace_indicator_scores(ev)library(r4subcore) ctx <- r4sub_run_context(study_id = "TEST001", environment = "DEV") adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGE", "AGEGR1"), label = c("Study ID", "Age", "Age Group") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) ev <- trace_model_to_evidence(tm, ctx = ctx) trace_indicator_scores(ev)
Emits evidence rows compatible with r4subcore::validate_evidence() for
each ADaM variable's trace level, plus diagnostic rows for orphans,
ambiguities, and conflicts.
trace_model_to_evidence( trace_model, ctx, source_name = "r4subtrace", source_version = NULL )trace_model_to_evidence( trace_model, ctx, source_name = "r4subtrace", source_version = NULL )
trace_model |
A |
ctx |
An |
source_name |
Character; the name of the evidence source. |
source_version |
Character or |
A data.frame of evidence rows passing r4subcore::validate_evidence().
library(r4subcore) ctx <- r4sub_run_context(study_id = "TEST001", environment = "DEV") adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) ev <- trace_model_to_evidence(tm, ctx = ctx) r4subcore::validate_evidence(ev)library(r4subcore) ctx <- r4sub_run_context(study_id = "TEST001", environment = "DEV") adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) sdtm_meta <- data.frame( dataset = "DM", variable = c("STUDYID", "AGE"), label = c("Study ID", "Age") ) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "AGE"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "AGE") ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) ev <- trace_model_to_evidence(tm, ctx = ctx) r4subcore::validate_evidence(ev)
A full coverage table lists every variable; on a real submission with tens of
thousands of variables that is not what a team consumes. This returns only the
problems: variables that are not traced to the required level, and, when
adam_derivations is supplied, derivation chains that never reach SDTM, that
are circular, or that were cut off at the depth limit. One variable can appear
more than once when it has more than one kind of problem.
trace_problems( trace_model, traced_min_level = 2L, adam_derivations = NULL, trace_depth = 3L )trace_problems( trace_model, traced_min_level = 2L, adam_derivations = NULL, trace_depth = 3L )
trace_model |
A |
traced_min_level |
The minimum trace level that counts as traced, an
integer in |
adam_derivations |
Optional ADaM-to-ADaM derivation table (see
|
trace_depth |
Maximum chain depth to follow, passed to |
A tibble with one row per problem: adam_dataset, adam_var,
problem ("untraced", "ungrounded", "cycle", or "truncated"), and a
detail string. Empty when nothing is wrong.
traceability_coverage() for the full per-dataset rollup,
trace_chains() for the chains themselves.
adam_meta <- data.frame( dataset = "ADSL", variable = c("USUBJID", "AGE", "DERIVED1"), label = c("Unique Subject ID", "Age", "Derived") ) sdtm_meta <- data.frame(dataset = "DM", variable = c("USUBJID", "AGE")) map <- data.frame( adam_dataset = "ADSL", adam_var = c("USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("USUBJID", "AGE"), confidence = c(1, 1) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) trace_problems(tm)adam_meta <- data.frame( dataset = "ADSL", variable = c("USUBJID", "AGE", "DERIVED1"), label = c("Unique Subject ID", "Age", "Derived") ) sdtm_meta <- data.frame(dataset = "DM", variable = c("USUBJID", "AGE")) map <- data.frame( adam_dataset = "ADSL", adam_var = c("USUBJID", "AGE"), sdtm_domain = "DM", sdtm_var = c("USUBJID", "AGE"), confidence = c(1, 1) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) trace_problems(tm)
Summarizes how much of each ADaM dataset is traced back to SDTM. A variable
counts as traced when its trace level reaches traced_min_level. The default
of 2 treats an existing SDTM mapping (level L2) as traced.
traceability_coverage(trace_model, traced_min_level = 2L)traceability_coverage(trace_model, traced_min_level = 2L)
trace_model |
A |
traced_min_level |
The minimum trace level that counts as traced, an
integer in |
A tibble with one row per ADaM dataset: adam_dataset,
n_variables, n_traced, n_untraced, pct_traced, and untraced_vars
(a comma-separated list, empty when the dataset is fully traced).
adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGEGR1"), label = c("Study ID", "Unique Subject ID", NA) ) sdtm_meta <- data.frame(dataset = "DM", variable = c("STUDYID", "USUBJID")) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID"), confidence = c(1, 1) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) traceability_coverage(tm)adam_meta <- data.frame( dataset = "ADSL", variable = c("STUDYID", "USUBJID", "AGEGR1"), label = c("Study ID", "Unique Subject ID", NA) ) sdtm_meta <- data.frame(dataset = "DM", variable = c("STUDYID", "USUBJID")) map <- data.frame( adam_dataset = "ADSL", adam_var = c("STUDYID", "USUBJID"), sdtm_domain = "DM", sdtm_var = c("STUDYID", "USUBJID"), confidence = c(1, 1) ) tm <- build_trace_model(adam_meta, sdtm_meta, mapping = map) traceability_coverage(tm)
Checks that a mapping data.frame contains the required columns
(adam_dataset, adam_var, sdtm_domain, sdtm_var) and canonicalizes
names, trims whitespace, and optionally uppercases dataset/domain names.
validate_mapping(df, uppercase_datasets = TRUE)validate_mapping(df, uppercase_datasets = TRUE)
df |
A data.frame describing ADaM-to-SDTM variable mappings. |
uppercase_datasets |
Logical; if |
A tibble with canonicalized column names and values.
map <- data.frame( ADAM_DATASET = "adsl", ADAM_VAR = "AGE", SDTM_DOMAIN = "dm", SDTM_VAR = "AGE" ) validate_mapping(map)map <- data.frame( ADAM_DATASET = "adsl", ADAM_VAR = "AGE", SDTM_DOMAIN = "dm", SDTM_VAR = "AGE" ) validate_mapping(map)
Checks that an ADaM or SDTM metadata data.frame contains the required
columns (dataset, variable) and canonicalizes column names to lowercase.
validate_metadata(df, kind = c("adam", "sdtm"))validate_metadata(df, kind = c("adam", "sdtm"))
df |
A data.frame of dataset metadata. |
kind |
Character; |
A tibble with canonicalized column names.
meta <- data.frame(DATASET = "ADSL", VARIABLE = "SUBJID", LABEL = "Subject ID") validate_metadata(meta, kind = "adam")meta <- data.frame(DATASET = "ADSL", VARIABLE = "SUBJID", LABEL = "Subject ID") validate_metadata(meta, kind = "adam")