GTensor API

The GTensor module for genomic tensor analysis. This module provides functionality for creating, manipulating, and analyzing genomic tensors, including loading datasets, applying transformations, and generating explanations for model components.

GTensors are hierarchical, multi-dimensional arrays designed to represent complex genomic data structures. They are sliceable along multiple dimensions, and support lazy loading for memory efficiency.

Use the Gtensor CLI tool to interact with and build GTensor datasets from the command line - the python API is mostly intended for analysis and visualization.

Below is an interactive example of using GTensors in a Jupyter notebook:

<xarray.Dataset> Size: 1GB
Dimensions:                         (configuration: 2, locus: 55220,
                                     context: 96, component: 18,
                                     shap_features: 19, shap_component: 18,
                                     shap_locus: 2000, genome_state: 5,
                                     feature: 19, sample: 736,
                                     mesoscale_state: 9)
Coordinates:
  * configuration                   (configuration) <U12 96B 'C/T-centered' '...
  * locus                           (locus) int64 442kB 2 3 4 ... 65746 65747
  * context                         (context) <U7 3kB 'A[C>A]A' ... 'T[T>C]T'
  * component                       (component) <U3 216B 'M0' 'M1' ... 'M17'
  * shap_features                   (shap_features) <U15 1kB 'ATACAccessible'...
  * shap_component                  (shap_component) <U3 216B 'M0' ... 'M17'
  * shap_locus                      (shap_locus) int64 16kB 24008 ... 22805
  * genome_state                    (genome_state) <U30 600B 'Baseline' ... '...
  * feature                         (feature) <U15 1kB 'ATACAccessible' ... '...
  * sample                          (sample) <U19 56kB 'BR001.purple' ... 'TC...
Dimensions without coordinates: mesoscale_state
Data variables: (12/48)
    Spectra/interactions            (component, genome_state, context) float64 69kB ...
    Spectra/shared_effects          (component, genome_state) float64 720B 0....
    Spectra/spectra                 (component, context, genome_state) float64 69kB ...
    Regions/chrom                   (locus) <U4 884kB 'chr1' 'chr1' ... 'chr1'
    Regions/start                   (locus) int64 442kB 819220 ... 248905601
    Regions/end                     (locus) int64 442kB 820420 ... 248906235
    ...                              ...
    State/mesoscale_idx             (configuration, locus) int64 884kB 4 4 ... 4
    State/log_context_distribution  (component, context, mesoscale_state) float64 124kB ...
    State/locus_features            (locus, feature) float32 4MB 0.0 ... -0.1123
    State/log_locus_distribution    (component, locus) float64 8MB -0.6555 .....
    ploidy                          (sample, locus) float32 0B <COO: nnz=0, fill_value=0.0>
    X                               (sample, configuration, context, locus) float32 329MB <GCXS: nnz=329103, fill_value=0.0>
Attributes:
    name:            breast
    dtype:           sbs
    genome_file:     /n/data1/hms/dbmi/park/ctDNA_loci_project/locusregressio...
    fasta_file:      /n/data1/hms/dbmi/park/SOFTWARE/REFERENCE/hg38/cgap_matc...
    blacklist_file:  /n/data1/hms/dbmi/park/ctDNA_loci_project/locusregressio...
    region_size:     10000
    filename:        retry2.nc
    regions_file:    ENFORM-Breast.nc.regions.bed

G-Tensor: breast

• Dimensions:
  • configuration: 2
  • locus: 55220
  • context: 96
  • component: 18
  • shap_features: 19
  • shap_component: 18
  • shap_locus: 2000
  • genome_state: 5
  • feature: 19
  • sample: 736
  • mesoscale_state: 9
• X: (1)
  • X
    (sample, configuration, context, locus)
    float32
    <GCXS: nnz=329103, fill_value=0.0>

    Format	gcxs
    Data Type	float32
    Shape	(736, 2, 96, 55220)
    nnz	329103
    Density	4.2175126691849206e-05
    Read-only	True
    Size	313.8M
    Storage ratio	0.01
    Compressed Axes	(0, 3)

• Data: (10)
  • component_distributions
    (component, locus, configuration, context)
    float32
    0.0008162 8.418e-05 ... 0.0004949

    array([[[[8.16181360e-04, 8.41755609e-05, 1.13639270e-03, ...,
              1.12953203e-04, 1.14218077e-04, 3.74800758e-04],
             [8.16181360e-04, 8.41755609e-05, 1.13639270e-03, ...,
              1.12953203e-04, 1.14218077e-04, 3.74800758e-04]],
    
            [[1.23187574e-03, 1.27047548e-04, 1.71517604e-03, ...,
              1.70482104e-04, 1.72391199e-04, 5.65692899e-04],
             [1.23187574e-03, 1.27047548e-04, 1.71517604e-03, ...,
              1.70482104e-04, 1.72391199e-04, 5.65692899e-04]],
    
            [[1.07886398e-03, 1.11266912e-04, 1.50213332e-03, ...,
              1.49306448e-04, 1.50978405e-04, 4.95427928e-04],
             [1.07886398e-03, 1.11266912e-04, 1.50213332e-03, ...,
              1.49306448e-04, 1.50978405e-04, 4.95427928e-04]],
    
            ...,
    
            [[2.28469167e-03, 2.35628031e-04, 3.18104192e-03, ...,
              3.16183694e-04, 3.19724379e-04, 1.04915921e-03],
             [2.28469167e-03, 2.35628031e-04, 3.18104192e-03, ...,
    ...
             [1.40159193e-03, 1.19357917e-03, 1.27034518e-03, ...,
              5.58595289e-04, 3.67310655e-04, 2.58523942e-04]],
    
            ...,
    
            [[1.19145133e-03, 1.01462600e-03, 1.07988238e-03, ...,
              4.74845147e-04, 3.12239776e-04, 2.19763460e-04],
             [1.19145133e-03, 1.01462600e-03, 1.07988238e-03, ...,
              4.74845147e-04, 3.12239776e-04, 2.19763460e-04]],
    
            [[1.13756000e-03, 9.68732696e-04, 1.03103754e-03, ...,
              4.53367102e-04, 2.98116647e-04, 2.09823193e-04],
             [1.13756000e-03, 9.68732696e-04, 1.03103754e-03, ...,
              4.53367102e-04, 2.98116647e-04, 2.09823193e-04]],
    
            [[2.68330425e-03, 2.28507025e-03, 2.43203621e-03, ...,
              1.06941327e-03, 7.03204831e-04, 4.94936015e-04],
             [2.68330425e-03, 2.28507025e-03, 2.43203621e-03, ...,
              1.06941327e-03, 7.03204831e-04, 4.94936015e-04]]]],
          dtype=float32)

  • component_distributions_locus
    (component, locus)
    float32
    0.001608 0.002517 ... 0.005958

    array([[0.00160757, 0.00251667, 0.00220055, ..., 0.00408187, 0.00212895,
            0.00060106],
           [0.00211766, 0.00232938, 0.00241099, ..., 0.00297932, 0.00211661,
            0.00163246],
           [0.00315005, 0.00206169, 0.00238894, ..., 0.00303443, 0.0032256 ,
            0.0049133 ],
           ...,
           [0.00109402, 0.00121836, 0.00130745, ..., 0.00434199, 0.00232904,
            0.00056757],
           [0.00105494, 0.0006052 , 0.00065257, ..., 0.00296408, 0.00129849,
            0.00066819],
           [0.00330446, 0.00255067, 0.00340437, ..., 0.00242862, 0.00213621,
            0.00595791]], dtype=float32)

  • contributions
    (sample, component)
    float64
    153.5 22.14 36.92 ... 53.72 95.67

    array([[ 153.4715177 ,   22.13884017,   36.91701085, ...,   86.70299598,
              33.24296395,   71.34830521],
           [  55.08241008,   49.05304707,   46.57326334, ...,   27.59948763,
             223.07354553,    8.71553456],
           [ 207.8377233 , 1058.60078083,  470.59836292, ...,  834.5530926 ,
              83.26043691,   94.11466044],
           ...,
           [ 146.0599038 ,  651.59639144,   37.42808488, ..., 1214.13365932,
              34.13908786,  140.71908414],
           [  45.7552755 ,   57.77040488,   15.67352332, ...,   80.22501747,
              16.02944648,   19.83405096],
           [  87.59643072,  157.00834041,  381.94371569, ...,  689.56504454,
              53.7202158 ,   95.67161757]])

  • predicted_marginal
    (locus, configuration, context)
    float32
    0.01061 0.01065 ... 0.007632

    array([[[0.01060738, 0.01065123, 0.01573445, ..., 0.0042932 ,
             0.00527245, 0.00569849],
            [0.01060738, 0.01065123, 0.01573445, ..., 0.0042932 ,
             0.00527245, 0.00569849]],
    
           [[0.01075247, 0.01121671, 0.0161343 , ..., 0.00424063,
             0.00546737, 0.00598755],
            [0.01075247, 0.01121671, 0.0161343 , ..., 0.00424063,
             0.00546737, 0.00598755]],
    
           [[0.01126103, 0.01180904, 0.01681324, ..., 0.00452318,
             0.00580547, 0.00639821],
            [0.01126103, 0.01180904, 0.01681324, ..., 0.00452318,
             0.00580547, 0.00639821]],
    
           ...,
    
           [[0.02176665, 0.01542303, 0.02608942, ..., 0.00907317,
             0.00785733, 0.0083876 ],
            [0.02176665, 0.01542303, 0.02608942, ..., 0.00907317,
             0.00785733, 0.0083876 ]],
    
           [[0.01427104, 0.01292776, 0.01901028, ..., 0.00607623,
             0.0063722 , 0.00714005],
            [0.01427104, 0.01292776, 0.01901028, ..., 0.00607623,
             0.0063722 , 0.00714005]],
    
           [[0.00901599, 0.01133972, 0.0160096 , ..., 0.00440132,
             0.00667007, 0.00763244],
            [0.00901599, 0.01133972, 0.0160096 , ..., 0.00440132,
             0.00667007, 0.00763244]]], dtype=float32)

  • predicted_marginal_locus
    (locus)
    float32
    0.002028 0.001898 ... 0.002174

    array([0.00202793, 0.0018982 , 0.00206177, ..., 0.00297863, 0.00226454,
           0.00217435], dtype=float32)

  • empirical_marginal
    (configuration, context, locus)
    float32
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    array([[[0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.003513, 0.      ,
             0.      ],
            ...,
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ]],
    
           [[0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            ...,
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ],
            [0.      , 0.      , 0.      , ..., 0.      , 0.      ,
             0.      ]]], dtype=float32)

  • empirical_locus_marginal
    (locus)
    float32
    0.0 0.0 ... 0.000908 0.001367

    array([0.        , 0.        , 0.00043093, ..., 0.00083119, 0.00090803,
           0.00136667], dtype=float32)

  • SHAP_values
    (shap_component, shap_locus, shap_features)
    float64
    0.1228 0.3217 ... 0.4938 -0.6256

    array([[[ 1.22837441e-01,  3.21652283e-01,  7.09834633e-01, ...,
              1.36791615e-01,  1.90237902e-01,  1.16650823e-01],
            [ 1.65678441e-01, -4.21049857e-01,  4.82336636e-01, ...,
             -1.09218084e-01,  1.14808520e-02, -8.66587654e-02],
            [ 1.82902677e-01, -6.14474052e-02,  7.10553294e-01, ...,
             -4.09953165e-02,  2.08118112e-02, -7.88478043e-03],
            ...,
            [ 1.29081442e-01, -8.92306592e-02, -2.04736338e-01, ...,
             -1.13833142e-01, -9.53741930e-02, -3.22041866e-01],
            [ 1.12114460e-01,  8.12029929e-03, -3.07306229e-01, ...,
              4.66200991e-02,  8.17315067e-02, -2.68343771e-02],
            [ 1.42439715e-01,  1.05133274e-02, -2.66116744e-02, ...,
             -7.87383187e-02,  5.00472210e-02,  3.91414906e-02]],
    
           [[ 6.02092726e-02, -1.82894087e-01,  3.97642206e-01, ...,
              2.03133977e-02, -4.21978677e-01, -2.42697578e-01],
            [ 8.75111640e-02,  7.95553347e-02,  5.87230599e-01, ...,
             -2.92938814e-02,  2.48315784e-01,  5.56756099e-01],
            [ 9.14071224e-02,  1.77469764e-01,  6.09212858e-01, ...,
             -1.27621031e-03, -1.91555363e-01, -8.24380553e-02],
    ...
            [ 9.16641335e-03, -1.16597522e+00, -2.67470273e-01, ...,
             -8.30427214e-01, -1.41603057e-01, -5.23739703e-01],
            [ 5.57369250e-03,  1.41663399e+00,  4.55734196e-01, ...,
              8.95244120e-01,  2.58170102e-01, -4.59576810e-01],
            [ 6.96594935e-03, -6.73727659e-01,  1.00855999e+00, ...,
             -1.46101942e-01, -1.43379558e-01, -1.89844717e-01]],
    
           [[ 1.06622343e+00, -2.15937494e-01,  8.52110860e-01, ...,
              1.60207648e-01,  4.25544104e-01,  8.92412507e-01],
            [ 4.25154191e-01,  3.89179708e-01,  1.80773071e-01, ...,
             -2.49177752e-01, -6.60704054e-01, -1.51076905e+00],
            [ 5.74709662e-01,  6.13348497e-01,  7.13975670e-01, ...,
              1.64788821e-01,  3.60036322e-01,  6.84919343e-02],
            ...,
            [ 6.91545298e-01, -1.95004554e-01, -8.49691902e-02, ...,
             -2.70138959e-01,  2.42302979e-01, -3.00372646e+00],
            [ 9.72820413e-01, -4.99695516e-02, -3.72846840e-01, ...,
              2.83388820e-01,  4.32839110e-01,  1.00508484e+00],
            [ 8.53362136e-01, -2.09299562e-02, -8.22562016e-01, ...,
              1.82472169e-01,  4.93805831e-01, -6.25623146e-01]]])

  • empirical_marginal_locus
    (locus)
    float32
    0.0 0.0 ... 0.000908 0.001367

    array([0.        , 0.        , 0.00043093, ..., 0.00083119, 0.00090804,
           0.00136666], dtype=float32)

  • ploidy
    (sample, locus)
    float32
    <COO: nnz=0, fill_value=0.0>

    Format	coo
    Data Type	float32
    Shape	(736, 55220)
    nnz	0
    Density	0.0
    Read-only	True
    Size	0
    Storage ratio	0.00

• Features: (22)
  • Features/GenePosition
    (locus)
    float32
    nan nan nan nan ... nan nan 0.0 0.0

    normalization :

    minmax

    group :

    all

    active :

    1

    array([nan, nan, nan, ..., nan,  0.,  0.], dtype=float32)

  • Features/GeneExpression
    (locus)
    float32
    nan nan nan nan ... nan nan nan nan

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([nan, nan, nan, ..., nan, nan, nan], dtype=float32)

  • Features/NucleotideRatio
    (locus)
    float32
    0.3104 0.3104 ... 0.251 0.251

    normalization :

    standardize

    group :

    all

    active :

    1

    array([0.310445 , 0.310445 , 0.310445 , ..., 0.3088985, 0.25105  ,
           0.25105  ], dtype=float32)

  • Features/BackgroundMutrate
    (locus)
    float32
    4.018e-07 4.821e-07 ... 1.125e-06

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([4.017535e-07, 4.821040e-07, 4.821040e-07, ..., 1.721800e-06,
           1.320045e-06, 1.124910e-06], dtype=float32)

  • Features/ATACAccessible
    (locus)
    <U4
    'none' 'none' ... 'none' 'none'

    normalization :

    categorical

    group :

    all

    active :

    1

    classes :

    ['none', 'Acc']

    array(['none', 'none', 'none', ..., 'none', 'none', 'none'], dtype='<U4')

  • Features/GeneStrand
    (locus)
    int8
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    normalization :

    strand

    group :

    all

    active :

    1

    array([0, 0, 0, ..., 0, 0, 0], dtype=int8)

  • Features/ReplicationStrand
    (locus)
    int8
    0 0 0 0 1 1 1 1 ... 0 0 0 0 0 0 0 0

    normalization :

    strand

    group :

    all

    active :

    1

    array([0, 0, 0, ..., 0, 0, 0], dtype=int8)

  • Features/RepliseqG1b
    (locus)
    float32
    51.0 51.0 52.25 ... 8.336 7.45 7.0

    normalization :

    quantile

    group :

    all

    active :

    1

    array([51.     , 51.     , 52.2504 , ...,  8.33616,  7.45013,  7.     ],
          dtype=float32)

  • Features/HICEigenvector
    (locus)
    float32
    nan 0.8588 0.8588 ... 0.5943 0.5943

    normalization :

    standardize

    group :

    all

    active :

    1

    array([     nan, 0.858771, 0.858771, ..., 0.545264, 0.594336, 0.594336],
          dtype=float32)

  • Features/RepliseqS3
    (locus)
    float32
    4.0 4.0 4.0 ... 25.66 26.0 26.0

    normalization :

    quantile

    group :

    all

    active :

    1

    array([ 4.    ,  4.    ,  4.    , ..., 25.6638, 26.    , 26.    ],
          dtype=float32)

  • Features/RepliseqS2
    (locus)
    float32
    6.0 6.0 6.0 ... 35.0 34.58 35.0

    normalization :

    quantile

    group :

    all

    active :

    1

    array([ 6.    ,  6.    ,  6.    , ..., 35.    , 34.5773, 35.    ],
          dtype=float32)

  • Features/RepliseqS1
    (locus)
    float32
    28.75 28.0 27.46 ... 15.97 16.0

    normalization :

    quantile

    group :

    all

    active :

    1

    array([28.7492, 28.    , 27.4639, ..., 15.2016, 15.9725, 16.    ],
          dtype=float32)

  • Features/RepliseqS4
    (locus)
    float32
    2.0 2.0 2.0 2.152 ... 8.933 9.0 9.0

    normalization :

    quantile

    group :

    all

    active :

    1

    array([2.     , 2.     , 2.     , ..., 8.93288, 9.     , 9.     ],
          dtype=float32)

  • Features/RepliseqG2
    (locus)
    float32
    9.0 9.0 8.107 ... 7.202 7.0 7.0

    group :

    all

    normalization :

    quantile

    active :

    1

    array([9.     , 9.     , 8.10679, ..., 7.20164, 7.     , 7.     ],
          dtype=float32)

  • Features/DNase
    (locus)
    float32
    0.0888 0.1062 ... 0.03143 0.9611

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.0887982 , 0.1061594 , 0.1026488 , ..., 0.03864325, 0.0314343 ,
           0.961084  ], dtype=float32)

  • Features/H3K4me3
    (locus)
    float32
    0.2787 0.1536 ... 0.179 105.2

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([  0.2786615 ,   0.153628  ,   0.11353715, ...,   0.1348505 ,
             0.1790165 , 105.2045    ], dtype=float32)

  • Features/H3K27ac
    (locus)
    float32
    1.967 0.8837 2.086 ... 0.4331 55.81

    group :

    all

    normalization :

    log1p_cpm

    active :

    1

    array([ 1.967099 ,  0.8837265,  2.086472 , ...,  0.099964 ,  0.4330815,
           55.80805  ], dtype=float32)

  • Features/H3K27me3
    (locus)
    float32
    0.06838 0.1301 ... 0.1438 0.08787

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.0683846 , 0.13008985, 0.1398076 , ..., 0.694179  , 0.14381555,
           0.087869  ], dtype=float32)

  • Features/POLR2A
    (locus)
    float32
    0.2154 0.1579 ... 0.5409 5.66

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.2154145, 0.157861 , 0.1368415, ..., 0.3974495, 0.540921 ,
           5.65999  ], dtype=float32)

  • Features/H3K36me3
    (locus)
    float32
    0.5142 0.4556 ... 0.1691 0.07347

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.5141509 , 0.45561826, 0.3234962 , ..., 0.1789847 , 0.16908455,
           0.07346585], dtype=float32)

  • Features/H3K4me1
    (locus)
    float32
    0.4003 0.2917 ... 0.2302 0.484

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.400333 , 0.2917165, 0.4549685, ..., 0.148378 , 0.230199 ,
           0.483956 ], dtype=float32)

  • Features/H3K9me3
    (locus)
    float32
    0.1035 0.03242 ... 0.1344 0.1639

    normalization :

    log1p_cpm

    group :

    all

    active :

    1

    array([0.10352915, 0.03241775, 0.06453654, ..., 0.26354   , 0.1344295 ,
           0.16391996], dtype=float32)

• Regions: (6)
  • Regions/chrom
    (locus)
    <U4
    'chr1' 'chr1' ... 'chr1' 'chr1'

    array(['chr1', 'chr1', 'chr1', ..., 'chr1', 'chr1', 'chr1'], dtype='<U4')

  • Regions/start
    (locus)
    int64
    819220 820420 ... 248905601

    array([   819220,    820420,    820820, ..., 248895801, 248903235,
           248905601])

  • Regions/end
    (locus)
    int64
    820420 820820 ... 248906235

    array([   820420,    820820,    823620, ..., 248903235, 248905601,
           248906235])

  • Regions/length
    (locus)
    float32
    1.162e+03 400.0 ... 2.366e+03 262.0

    array([1162.,  400., 2755., ..., 7434., 2366.,  262.], dtype=float32)

  • Regions/context_frequencies
    (configuration, locus, context)
    float32
    28.0 28.0 28.0 39.0 ... 7.0 7.0 7.0

    array([[[ 28.,  28.,  28., ...,  33.,  33.,  33.],
            [ 12.,  12.,  12., ...,   0.,   0.,   0.],
            [ 54.,  54.,  54., ...,   3.,   3.,   3.],
            ...,
            [202., 202., 202., ..., 154., 154., 154.],
            [ 48.,  48.,  48., ..., 131., 131., 131.],
            [  9.,   9.,   9., ...,   4.,   4.,   4.]],
    
           [[ 31.,  31.,  31., ...,  29.,  29.,  29.],
            [ 13.,  13.,  13., ...,   2.,   2.,   2.],
            [ 92.,  92.,  92., ...,   2.,   2.,   2.],
            ...,
            [ 65.,  65.,  65., ..., 441., 441., 441.],
            [ 45.,  45.,  45., ...,  64.,  64.,  64.],
            [  5.,   5.,   5., ...,   7.,   7.,   7.]]], dtype=float32)

  • Regions/exposures
    (locus)
    float64
    1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0

    array([1., 1., 1., ..., 1., 1., 1.])

• Spectra: (3)
  • Spectra/interactions
    (component, genome_state, context)
    float64
    0.0 0.0 ... 0.0008534 -9.001e-08

    array([[[ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00, ...,
              0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
            [ 1.25569103e-01, -2.74438981e-23,  8.68699530e-02, ...,
              2.20584828e-12,  8.35952131e-17,  9.79064595e-03],
            [-6.56238341e-03,  6.76502435e-24,  8.03395748e-03, ...,
              1.85244925e-07,  3.24932040e-04, -3.17103631e-03],
            [-9.24484739e-04,  2.01858764e-23,  3.36079264e-01, ...,
             -7.32777792e-07,  1.79425523e-07, -3.06774761e-03],
            [ 3.44656300e-03, -3.02597082e-23, -4.08927387e-02, ...,
              7.15252988e-08, -6.92222654e-11, -4.48850561e-03]],
    
           [[ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00, ...,
              0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
            [ 1.67497975e-01,  1.20315268e-17, -3.04329335e-03, ...,
             -2.45456205e-03,  2.49722668e-06,  1.44398216e-18],
            [-3.79539545e-03, -2.78843280e-23,  2.18068398e-03, ...,
              1.27378431e-03,  8.24853347e-07,  2.10340286e-16],
            [ 7.00212708e-03,  5.83667693e-17,  5.52890968e-03, ...,
             -5.53263914e-04, -2.49157892e-04, -2.62695015e-14],
            [-1.26697786e-02,  7.07673800e-18, -1.16678549e-02, ...,
    ...
              0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
            [ 2.20674361e-04,  7.14350359e-03,  1.73748979e-02, ...,
              1.53943966e-02, -4.64778818e-02, -9.79529296e-04],
            [ 8.97633471e-03,  1.93836952e-03, -9.32580290e-04, ...,
              2.60083041e-02, -4.47075626e-02, -1.23101987e-02],
            [-2.18246542e-03, -3.06214586e-03, -1.37384086e-03, ...,
             -4.78083879e-03, -4.91988887e-03,  9.62061407e-03],
            [-7.45466329e-03, -6.75784227e-03,  2.53944276e-03, ...,
             -3.60595545e-03, -7.93239064e-02, -2.39427717e-02]],
    
           [[ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00, ...,
              0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
            [ 2.01493260e-02, -6.74117575e-03,  4.49657345e-04, ...,
              6.22534381e-06,  2.40479726e-06,  1.57516103e-21],
            [-4.06513590e-03,  1.08688003e-03,  2.60087433e-03, ...,
              1.64279615e-03,  5.32233151e-08, -5.03420910e-21],
            [ 4.56564503e-03,  4.10397702e-03,  1.01609419e-02, ...,
              1.05784944e-03, -1.09036055e-04, -1.76025675e-08],
            [-3.81221504e-03,  2.86408643e-03, -2.90163535e-02, ...,
             -1.77414161e-03,  8.53381541e-04, -9.00125601e-08]]])

  • Spectra/shared_effects
    (component, genome_state)
    float64
    0.0 -0.0481 ... 0.1101 0.03108

    array([[ 0.00000000e+00, -4.81037849e-02, -4.32578066e-01,
             1.08163034e-02, -7.58048884e-02],
           [ 0.00000000e+00,  1.95227399e-02, -1.48470488e-01,
            -7.34079774e-03,  1.22778641e-01],
           [ 0.00000000e+00, -3.04799516e-02, -4.26108472e-03,
             2.72764546e-01, -2.76172137e-01],
           [ 0.00000000e+00, -1.36491367e-01,  9.69064655e-04,
             6.37503060e-04,  3.11612632e-02],
           [ 0.00000000e+00, -1.53179445e-01, -9.74470570e-02,
             7.62877623e-03,  2.47156700e-03],
           [ 0.00000000e+00, -1.06492720e-01,  2.43080347e-01,
             1.58301847e-01, -2.00599253e-01],
           [ 0.00000000e+00, -1.68604766e-02, -1.22141689e-01,
             1.64560306e-02, -6.10448287e-02],
           [ 0.00000000e+00,  4.76205361e-01, -3.26899981e-01,
             7.14483446e-03,  4.63638046e-02],
           [ 0.00000000e+00,  1.89880476e-01,  7.02495183e-01,
            -8.66828603e-04,  3.61914412e-02],
           [ 0.00000000e+00, -9.40706420e-02, -9.13451694e-02,
             1.03579797e-02, -5.44486202e-02],
           [ 0.00000000e+00, -1.89652235e-01, -1.92262769e-01,
            -1.67592295e-02,  9.03654336e-02],
           [ 0.00000000e+00, -1.32736017e-01, -7.45286526e-02,
            -7.11487209e-02,  2.00017622e-03],
           [ 0.00000000e+00, -5.98592042e-03, -9.64007254e-03,
             1.44524823e-01, -2.26558323e-01],
           [ 0.00000000e+00,  6.56795525e-02, -2.49137932e-04,
             1.86812300e-01, -1.04978415e-01],
           [ 0.00000000e+00,  1.15024217e-01, -1.30468799e-01,
            -5.14399775e-04, -2.25594875e-02],
           [ 0.00000000e+00, -1.35600683e-01, -1.85858054e-01,
            -4.88317332e-03,  1.56289782e-03],
           [ 0.00000000e+00, -5.06422372e-02, -1.28872607e-01,
             1.37195055e-02, -4.17557882e-02],
           [ 0.00000000e+00, -3.81150450e-02, -1.23054178e-01,
             1.10106366e-01,  3.10751807e-02]])

  • Spectra/spectra
    (component, context, genome_state)
    float64
    1.221 1.319 0.7871 ... 7.633 7.053

    array([[[1.22106728e+00, 1.31941784e+00, 7.87088049e-01, 1.23320579e+00,
             1.13583376e+00],
            [1.25932828e-01, 1.20018376e-01, 8.17095254e-02, 1.27302349e-01,
             1.16739361e-01],
            [1.70012707e+00, 1.76732864e+00, 1.11199855e+00, 2.40511703e+00,
             1.51286525e+00],
            ...,
            [3.24354134e-01, 3.09120801e-01, 2.10452094e-01, 3.27881249e-01,
             3.00675349e-01],
            [3.27986311e-01, 3.12582392e-01, 2.12877895e-01, 3.31553225e-01,
             3.04042345e-01],
            [1.07627024e+00, 1.03581488e+00, 6.96110046e-01, 1.08464217e+00,
             9.93231202e-01]],
    
           [[1.16187166e+03, 1.40081319e+03, 9.97768810e+02, 1.16147823e+03,
             1.29711305e+03],
            [1.11635767e+02, 1.13836616e+02, 9.62328718e+01, 1.10819272e+02,
             1.26219207e+02],
            [4.43026885e+02, 4.50388211e+02, 3.82734162e+02, 4.42224897e+02,
             4.95090831e+02],
    ...
            [1.28919656e+00, 1.24454669e+00, 1.16317682e+00, 1.30077191e+00,
             1.23202290e+00],
            [9.78361730e+00, 8.87811415e+00, 8.22459547e+00, 9.87008928e+00,
             8.66792636e+00],
            [1.53090515e+00, 1.45388226e+00, 1.32933173e+00, 1.56705692e+00,
             1.43355963e+00]],
    
           [[1.93122719e+01, 1.89684112e+01, 1.70069389e+01, 2.16588199e+01,
             1.98460245e+01],
            [1.64461033e+01, 1.57246940e+01, 1.45577176e+01, 1.84358827e+01,
             1.70138520e+01],
            [1.75038474e+01, 1.68568199e+01, 1.55174842e+01, 1.97408090e+01,
             1.75399219e+01],
            ...,
            [1.47732826e+01, 1.42208827e+01, 1.30842463e+01, 1.65103017e+01,
             1.52125593e+01],
            [9.71433880e+00, 9.35106631e+00, 8.58957189e+00, 1.08438744e+01,
             1.00295084e+01],
            [6.83723482e+00, 6.58153723e+00, 6.04559076e+00, 7.63306702e+00,
             7.05303820e+00]]])

• State: (6)
  • State/normalizers
    (component)
    float64
    -26.67 -33.09 ... -27.71 -29.11

    array([-26.67246946, -33.08705993, -28.14577253, -27.23268498,
           -30.92115067, -27.58819187, -29.03191126, -26.21341394,
           -26.47218838, -29.03107874, -27.23170241, -29.29682437,
           -24.88008036, -25.88154948, -25.80861274, -26.32767872,
           -27.71076313, -29.11402454])

  • State/topic_compositions
    (component, sample)
    float64
    0.9866 1.31 0.9686 ... 1.024 0.8428

    array([[0.98660812, 1.30991655, 0.96858276, ..., 1.20892512, 0.8542318 ,
            1.05708114],
           [0.98219968, 1.04085065, 1.00867219, ..., 0.88416297, 0.93619271,
            1.11542159],
           [0.99437413, 0.97373543, 1.08971617, ..., 0.85258992, 1.00881259,
            1.13399989],
           ...,
           [0.99581784, 0.80668588, 1.04100159, ..., 1.10980655, 1.04172925,
            1.04570226],
           [1.08037849, 0.8791703 , 1.18917993, ..., 1.00574662, 0.9642266 ,
            1.11717325],
           [1.06058621, 0.99303629, 0.80344021, ..., 0.94449919, 1.02351127,
            0.84283685]])

  • State/mesoscale_idx
    (configuration, locus)
    int64
    4 4 4 4 5 5 5 5 ... 4 4 4 4 4 4 4 4

    array([[4, 4, 4, ..., 4, 4, 4],
           [4, 4, 4, ..., 4, 4, 4]])

  • State/log_context_distribution
    (component, context, mesoscale_state)
    float64
    4.596 4.586 4.514 ... 5.456 5.488

    array([[[ 4.5962665 ,  4.58637469,  4.51401636, ...,  4.07966074,
              4.06976892,  3.99741059],
            [ 2.19988997,  2.18907366,  2.11326877, ...,  1.81541568,
              1.80459938,  1.72879449],
            [ 5.22554881,  4.87865324,  4.76195561, ...,  4.76223853,
              4.41534296,  4.29864534],
            ...,
            [ 2.4939582 ,  2.48314263,  2.40733781, ...,  2.1094841 ,
              2.09866853,  2.02286371],
            [ 2.50509506,  2.49427858,  2.41847369, ...,  2.12094571,
              2.11012923,  2.03432434],
            [ 3.70010277,  3.69235421,  3.61206082, ...,  3.3026668 ,
              3.29491825,  3.21462485]],
    
           [[11.5536536 , 11.55399227, 11.66410113, ..., 11.214367  ,
             11.21470567, 11.32481453],
            [ 9.03660751,  9.0439483 ,  9.16672694, ...,  8.86861428,
              8.87595508,  8.99873372],
            [10.41748209, 10.41929398, 10.53040476, ..., 10.25471284,
             10.25652473, 10.36763551],
    ...
            [ 3.88487579,  3.87593712,  3.83057538, ...,  3.81725933,
              3.80832066,  3.76295892],
            [ 5.84955455,  5.84075493,  5.71967523, ...,  5.77309449,
              5.76429488,  5.64321518],
            [ 4.05474328,  4.03140316,  3.9657046 , ...,  3.96518224,
              3.94184212,  3.87614356]],
    
           [[ 7.36663111,  7.2519591 ,  7.27922207, ...,  7.25747752,
              7.14280551,  7.17006847],
            [ 7.17862676,  7.06441642,  7.09835569, ...,  7.10151569,
              6.98730534,  7.02124461],
            [ 7.2542067 ,  7.13393939,  7.13599822, ...,  7.17141878,
              7.05115148,  7.0532103 ],
            ...,
            [ 6.42304148,  6.31187726,  6.3411783 , ...,  6.33973892,
              6.2285747 ,  6.25787574],
            [ 6.00265348,  5.89265615,  5.92458471, ...,  5.91771199,
              5.80771466,  5.83964322],
            [ 5.65154045,  5.54143411,  5.5725092 , ...,  5.56660132,
              5.45649497,  5.48757006]]])

  • State/locus_features
    (locus, feature)
    float32
    0.0 0.1433 nan ... 0.8378 -0.1123

    array([[ 0.        ,  0.14334883,         nan, ...,  0.37237236,
                    nan,  0.12899952],
           [ 0.        ,  0.3168423 ,         nan, ...,  0.37237236,
             1.3256534 ,  0.12899952],
           [ 0.        ,  0.28321943,         nan, ...,  0.37237236,
             1.3256534 ,  0.12899952],
           ...,
           [ 0.        , -0.49613106,         nan, ...,  0.6231408 ,
             0.7472138 ,  0.12271691],
           [ 0.        , -0.61278236,         nan, ...,  0.6261261 ,
             0.83775455, -0.11229002],
           [ 0.        ,  3.2285004 ,         nan, ...,  0.6261261 ,
             0.83775455, -0.11229002]], dtype=float32)

  • State/log_locus_distribution
    (component, locus)
    float64
    -0.6555 -0.2439 ... -0.643 0.2152

    array([[-0.65554925, -0.2438925 , -0.37652196, ...,  0.37380053,
            -0.22485962, -1.59230489],
           [-0.1627731 , -0.15052637, -0.1111911 , ...,  0.24952612,
            -0.05667807, -0.40514609],
           [ 0.01972296,  0.04950112,  0.06736122, ..., -0.04436633,
            -0.00244809,  0.48643181],
           ...,
           [-0.49480668, -0.46365254, -0.37882031, ...,  0.93277457,
             0.32210232, -1.15851299],
           [-0.6184988 , -0.77143883, -0.78516632, ...,  0.34161587,
            -0.52125367, -1.08819858],
           [-0.48760334, -0.66247436, -0.43428223, ..., -0.59671871,
            -0.64300528,  0.21515802]])

• Coordinates: (10)
  • configuration
    (configuration)
    <U12
    'C/T-centered' 'A/G-centered'

    array(['C/T-centered', 'A/G-centered'], dtype='<U12')

  • locus
    (locus)
    int64
    2 3 4 5 ... 65744 65745 65746 65747

    array([    2,     3,     4, ..., 65745, 65746, 65747])

  • context
    (context)
    <U7
    'A[C>A]A' 'A[C>G]A' ... 'T[T>C]T'

    array(['A[C>A]A', 'A[C>G]A', 'A[C>T]A', 'A[C>A]C', 'A[C>G]C', 'A[C>T]C',
           'A[C>A]G', 'A[C>G]G', 'A[C>T]G', 'A[C>A]T', 'A[C>G]T', 'A[C>T]T',
           'C[C>A]A', 'C[C>G]A', 'C[C>T]A', 'C[C>A]C', 'C[C>G]C', 'C[C>T]C',
           'C[C>A]G', 'C[C>G]G', 'C[C>T]G', 'C[C>A]T', 'C[C>G]T', 'C[C>T]T',
           'G[C>A]A', 'G[C>G]A', 'G[C>T]A', 'G[C>A]C', 'G[C>G]C', 'G[C>T]C',
           'G[C>A]G', 'G[C>G]G', 'G[C>T]G', 'G[C>A]T', 'G[C>G]T', 'G[C>T]T',
           'T[C>A]A', 'T[C>G]A', 'T[C>T]A', 'T[C>A]C', 'T[C>G]C', 'T[C>T]C',
           'T[C>A]G', 'T[C>G]G', 'T[C>T]G', 'T[C>A]T', 'T[C>G]T', 'T[C>T]T',
           'A[T>A]A', 'A[T>G]A', 'A[T>C]A', 'A[T>A]C', 'A[T>G]C', 'A[T>C]C',
           'A[T>A]G', 'A[T>G]G', 'A[T>C]G', 'A[T>A]T', 'A[T>G]T', 'A[T>C]T',
           'C[T>A]A', 'C[T>G]A', 'C[T>C]A', 'C[T>A]C', 'C[T>G]C', 'C[T>C]C',
           'C[T>A]G', 'C[T>G]G', 'C[T>C]G', 'C[T>A]T', 'C[T>G]T', 'C[T>C]T',
           'G[T>A]A', 'G[T>G]A', 'G[T>C]A', 'G[T>A]C', 'G[T>G]C', 'G[T>C]C',
           'G[T>A]G', 'G[T>G]G', 'G[T>C]G', 'G[T>A]T', 'G[T>G]T', 'G[T>C]T',
           'T[T>A]A', 'T[T>G]A', 'T[T>C]A', 'T[T>A]C', 'T[T>G]C', 'T[T>C]C',
           'T[T>A]G', 'T[T>G]G', 'T[T>C]G', 'T[T>A]T', 'T[T>G]T', 'T[T>C]T'],
          dtype='<U7')

  • component
    (component)
    <U3
    'M0' 'M1' 'M2' ... 'M16' 'M17'

    array(['M0', 'M1', 'M2', 'M3', 'M4', 'M5', 'M6', 'M7', 'M8', 'M9', 'M10',
           'M11', 'M12', 'M13', 'M14', 'M15', 'M16', 'M17'], dtype='<U3')

  • shap_features
    (shap_features)
    <U15
    'ATACAccessible' ... 'Nucleotide...

    array(['ATACAccessible', 'DNase', 'GeneExpression', 'H3K27ac', 'H3K27me3',
           'H3K36me3', 'H3K4me1', 'H3K4me3', 'H3K9me3', 'POLR2A', 'GenePosition',
           'RepliseqG1b', 'RepliseqG2', 'RepliseqS1', 'RepliseqS2', 'RepliseqS3',
           'RepliseqS4', 'HICEigenvector', 'NucleotideRatio'], dtype='<U15')

  • shap_component
    (shap_component)
    <U3
    'M0' 'M1' 'M2' ... 'M16' 'M17'

    array(['M0', 'M1', 'M2', 'M3', 'M4', 'M5', 'M6', 'M7', 'M8', 'M9', 'M10',
           'M11', 'M12', 'M13', 'M14', 'M15', 'M16', 'M17'], dtype='<U3')

  • shap_locus
    (shap_locus)
    int64
    24008 11878 29922 ... 51624 22805

    array([24008, 11878, 29922, ...,  1591, 51624, 22805])

  • genome_state
    (genome_state)
    <U30
    'Baseline' ... 'ReplicationStran...

    array(['Baseline', 'GeneStrand:A/G-centered', 'GeneStrand:C/T-centered',
           'ReplicationStrand:A/G-centered', 'ReplicationStrand:C/T-centered'],
          dtype='<U30')

  • feature
    (feature)
    <U15
    'ATACAccessible' ... 'Nucleotide...

    array(['ATACAccessible', 'DNase', 'GeneExpression', 'H3K27ac', 'H3K27me3',
           'H3K36me3', 'H3K4me1', 'H3K4me3', 'H3K9me3', 'POLR2A', 'GenePosition',
           'RepliseqG1b', 'RepliseqG2', 'RepliseqS1', 'RepliseqS2', 'RepliseqS3',
           'RepliseqS4', 'HICEigenvector', 'NucleotideRatio'], dtype='<U15')

  • sample
    (sample)
    <U19
    'BR001.purple' ... 'TCGA-GM-A3XL...

    array(['BR001.purple', 'BR002.purple', 'BR003.purple', ...,
           'TCGA-GI-A2C9.purple', 'TCGA-GM-A2DF.purple', 'TCGA-GM-A3XL.purple'],
          dtype='<U19')

• Indexes: (10)
  • configuration
    PandasIndex

    PandasIndex(Index(['C/T-centered', 'A/G-centered'], dtype='object', name='configuration'))

  • locus
    PandasIndex

    PandasIndex(Index([    2,     3,     4,     5,     6,     7,     8,     9,    12,    13,
           ...
           65737, 65738, 65739, 65740, 65741, 65743, 65744, 65745, 65746, 65747],
          dtype='int64', name='locus', length=55220))

  • context
    PandasIndex

    PandasIndex(Index(['A[C>A]A', 'A[C>G]A', 'A[C>T]A', 'A[C>A]C', 'A[C>G]C', 'A[C>T]C',
           'A[C>A]G', 'A[C>G]G', 'A[C>T]G', 'A[C>A]T', 'A[C>G]T', 'A[C>T]T',
           'C[C>A]A', 'C[C>G]A', 'C[C>T]A', 'C[C>A]C', 'C[C>G]C', 'C[C>T]C',
           'C[C>A]G', 'C[C>G]G', 'C[C>T]G', 'C[C>A]T', 'C[C>G]T', 'C[C>T]T',
           'G[C>A]A', 'G[C>G]A', 'G[C>T]A', 'G[C>A]C', 'G[C>G]C', 'G[C>T]C',
           'G[C>A]G', 'G[C>G]G', 'G[C>T]G', 'G[C>A]T', 'G[C>G]T', 'G[C>T]T',
           'T[C>A]A', 'T[C>G]A', 'T[C>T]A', 'T[C>A]C', 'T[C>G]C', 'T[C>T]C',
           'T[C>A]G', 'T[C>G]G', 'T[C>T]G', 'T[C>A]T', 'T[C>G]T', 'T[C>T]T',
           'A[T>A]A', 'A[T>G]A', 'A[T>C]A', 'A[T>A]C', 'A[T>G]C', 'A[T>C]C',
           'A[T>A]G', 'A[T>G]G', 'A[T>C]G', 'A[T>A]T', 'A[T>G]T', 'A[T>C]T',
           'C[T>A]A', 'C[T>G]A', 'C[T>C]A', 'C[T>A]C', 'C[T>G]C', 'C[T>C]C',
           'C[T>A]G', 'C[T>G]G', 'C[T>C]G', 'C[T>A]T', 'C[T>G]T', 'C[T>C]T',
           'G[T>A]A', 'G[T>G]A', 'G[T>C]A', 'G[T>A]C', 'G[T>G]C', 'G[T>C]C',
           'G[T>A]G', 'G[T>G]G', 'G[T>C]G', 'G[T>A]T', 'G[T>G]T', 'G[T>C]T',
           'T[T>A]A', 'T[T>G]A', 'T[T>C]A', 'T[T>A]C', 'T[T>G]C', 'T[T>C]C',
           'T[T>A]G', 'T[T>G]G', 'T[T>C]G', 'T[T>A]T', 'T[T>G]T', 'T[T>C]T'],
          dtype='object', name='context'))

  • component
    PandasIndex

    PandasIndex(Index(['M0', 'M1', 'M2', 'M3', 'M4', 'M5', 'M6', 'M7', 'M8', 'M9', 'M10',
           'M11', 'M12', 'M13', 'M14', 'M15', 'M16', 'M17'],
          dtype='object', name='component'))

  • shap_features
    PandasIndex

    PandasIndex(Index(['ATACAccessible', 'DNase', 'GeneExpression', 'H3K27ac', 'H3K27me3',
           'H3K36me3', 'H3K4me1', 'H3K4me3', 'H3K9me3', 'POLR2A', 'GenePosition',
           'RepliseqG1b', 'RepliseqG2', 'RepliseqS1', 'RepliseqS2', 'RepliseqS3',
           'RepliseqS4', 'HICEigenvector', 'NucleotideRatio'],
          dtype='object', name='shap_features'))

  • shap_component
    PandasIndex

    PandasIndex(Index(['M0', 'M1', 'M2', 'M3', 'M4', 'M5', 'M6', 'M7', 'M8', 'M9', 'M10',
           'M11', 'M12', 'M13', 'M14', 'M15', 'M16', 'M17'],
          dtype='object', name='shap_component'))

  • shap_locus
    PandasIndex

    PandasIndex(Index([24008, 11878, 29922, 58006, 41679, 19770, 47306, 65302, 34171, 15806,
           ...
           48720, 40978, 32794, 34818, 39979, 56282, 48461,  1591, 51624, 22805],
          dtype='int64', name='shap_locus', length=2000))

  • genome_state
    PandasIndex

    PandasIndex(Index(['Baseline', 'GeneStrand:A/G-centered', 'GeneStrand:C/T-centered',
           'ReplicationStrand:A/G-centered', 'ReplicationStrand:C/T-centered'],
          dtype='object', name='genome_state'))

  • feature
    PandasIndex

    PandasIndex(Index(['ATACAccessible', 'DNase', 'GeneExpression', 'H3K27ac', 'H3K27me3',
           'H3K36me3', 'H3K4me1', 'H3K4me3', 'H3K9me3', 'POLR2A', 'GenePosition',
           'RepliseqG1b', 'RepliseqG2', 'RepliseqS1', 'RepliseqS2', 'RepliseqS3',
           'RepliseqS4', 'HICEigenvector', 'NucleotideRatio'],
          dtype='object', name='feature'))

  • sample
    PandasIndex

    PandasIndex(Index(['BR001.purple', 'BR002.purple', 'BR003.purple', 'BR004.purple',
           'BR005.purple', 'BR006.purple', 'BR007.purple', 'BR008.purple',
           'BR009.purple', 'BR010.purple',
           ...
           'TCGA-E2-A1LL.purple', 'TCGA-E9-A1NH.purple', 'TCGA-EW-A1P8.purple',
           'TCGA-EW-A1PB.purple', 'TCGA-EW-A1PC.purple', 'TCGA-EW-A1PH.purple',
           'TCGA-EW-A3U0.purple', 'TCGA-GI-A2C9.purple', 'TCGA-GM-A2DF.purple',
           'TCGA-GM-A3XL.purple'],
          dtype='object', name='sample', length=736))

• Attributes: (8)

  name :

  breast

  dtype :

  sbs

  genome_file :

  /n/data1/hms/dbmi/park/ctDNA_loci_project/locusregression/gtensor_construction/reference_files/hg38/hg38.main-chroms.chromsizes

  fasta_file :

  /n/data1/hms/dbmi/park/SOFTWARE/REFERENCE/hg38/cgap_matches/Homo_sapiens_assembly38.fa

  blacklist_file :

  /n/data1/hms/dbmi/park/ctDNA_loci_project/locusregression/gtensor_construction/reference_files/hg38/blacklist_method12_v1_comb_sort_merged.no-chromends.bed.gz

  region_size :

  10000

  filename :

  retry2.nc

  regions_file :

  ENFORM-Breast.nc.regions.bed

GTensor API Reference

mutopia.gtensor.gtensor.GTensor(modality, *, name, chrom, start, end, context_frequencies, exposures=None, dtype=None)

Create a GTensor dataset for genomic tensor analysis.

This function constructs an xarray Dataset with the standardized structure required for genomic tensor operations, including region coordinates, context frequencies, and metadata.

Parameters:

• modality (object) – Modality object containing coordinate information and mode configuration

• name (str) – Name identifier for the dataset

• chrom (List[str]) – List of chromosome names for each genomic region

• start (List[int]) – List of start positions for each genomic region

• end (List[int]) – List of end positions for each genomic region

• context_frequencies (xr.DataArray) – Array containing context frequency data for each region

• exposures (Union[None, NDArray[np.number]], optional) – Exposure values for each region. If None, defaults to ones

• dtype (optional) – Data type for the dataset. If None, uses modality.MODE_ID

Returns:

Structured dataset with regions, coordinates, and metadata

Return type:

xr.Dataset

mutopia.gtensor.gtensor.annot_empirical_marginal(dataset, key='empirical_marginal')

Calculate and add empirical marginal mutation rates to a dataset.

This method computes empirical marginal mutation rates by aggregating observed mutations across all samples in the dataset and normalizing by context frequencies and region lengths.

Parameters:

• dataset (GTensorDataset) – Dataset containing mutation data to analyze

• key (str, default="empirical_marginal") – Base name for the mutation rate variables to be added to the dataset. Creates two variables: {key} and {key}_locus

Returns:

The input dataset with empirical marginal rates added as new variables: - {key}: Marginal mutation rates normalized by context frequencies - {key}_locus: Per-locus marginal rates normalized by region length

Return type:

GTensorDataset

mutopia.gtensor.gtensor.apply_to_samples(data, func, bar=True)

Apply a function to each sample in a dataset with parallel processing.

This function applies a given function to each sample (region) in the dataset, handling the parallelization and aggregation of results. It’s designed for operations that need to process each genomic region independently.

Parameters:

• data (GTensorDataset) – Input dataset or data loader containing samples to process

• func (callable) – Function to apply to each sample. Should accept a dataset slice and return a result that can be concatenated

• bar (bool, default=True) – Whether to display a progress bar during processing

Returns:

Dataset containing the concatenated results from all sample applications

Return type:

GTensorDataset

mutopia.gtensor.gtensor.dims_except_for(dims, *keepdims)

mutopia.gtensor.gtensor.eager_load(dataset)

Load a dataset eagerly with samples but without state information.

This is a convenience function that loads a dataset with sample data but excludes state information for faster access patterns.

Parameters:

dataset (str or path-like) – Path or identifier for the dataset to load

Returns:

Eager dataset interface with samples loaded into memory

Return type:

GTensorDataset

mutopia.gtensor.gtensor.eager_train_test_load(dataset, *test_chroms)

Load a dataset and perform eager train/test split by chromosomes.

This convenience function combines eager loading with train/test splitting, loading all data into memory for fast access.

Parameters:

• dataset (str or path-like) – Path or identifier for the dataset to load

• *test_chroms (str) – Chromosome names to reserve for the test set

Returns:

Training and testing dataset interfaces

Return type:

tuple[CorpusInterface, CorpusInterface]

mutopia.gtensor.gtensor.equal_size_quantiles(dataset, var_name, n_bins=10, key=None)

Create equal-size quantile bins for a variable in the dataset.

This function bins the values of a specified variable into quantiles of equal cumulative region length, which is useful for creating balanced genomic bins.

Parameters:

• dataset (GTensorDataset) – Dataset containing the variable to bin

• var_name (str) – Name of the variable to create quantile bins for

• n_bins (int, default=10) – Number of quantile bins to create

• key (str, optional) – Custom name for the output bin variable. If None, generates name as ‘{var_name_base}_qbins_{n_bins}’ where var_name_base is the last part of var_name after splitting on ‘/’

Returns:

The input dataset with quantile bins added as a new variable

Return type:

GTensorDataset

mutopia.gtensor.gtensor.excel_report(self, dataset, output, normalization='global')

Generate a comprehensive Excel report with model results.

This method creates an Excel file containing signature data, sample contributions, and SHAP values (if available) across multiple worksheets.

Parameters:

• dataset (GTensorDataset) – Dataset containing the model results to export

• output (str) – Output file path for the Excel report

Raises:

ImportError – If openpyxl is not installed for Excel writing support

Notes

The Excel file will contain the following sheets: - Signature_{name}: Normalized signature data for each component - Sample_contributions: Component contributions per sample (if available) - SHAP_transformed_features: SHAP feature data (if available) - SHAP_original_features: Original feature data for SHAP (if available) - SHAP_values_{component}: SHAP values for each component (if available)

Requires openpyxl to be installed: pip install openpyxl

mutopia.gtensor.gtensor.fetch_component(dataset, component_name)

Retrieve the mutational spectrum for a specific component.

This function extracts the signature spectrum (mutational profile) for a specified component from the dataset. The spectrum describes the relative frequency of different mutation types for this component.

Parameters:

• dataset (GTensorDataset) – Dataset containing component spectra

• component_name (Union[str, int]) – Name or index of the component to retrieve

Returns:

DataArray containing the component’s mutational spectrum with appropriate dimensions and coordinates

Return type:

xr.DataArray

Raises:

ValueError – If the specified component is not found in the dataset

mutopia.gtensor.gtensor.fetch_features(dataset, *feature_names, source=None)

Extract numerical features from the dataset’s “Features” section.

Parameters:

• dataset (GTensorDataset) – Dataset containing feature variables under the “Features” group.

• *feature_names (str) – Glob patterns or basenames of features to select. When empty, all numeric features are returned.

• source (str, optional) – Restrict selection to features within this source directory. When None, features from all sources are considered.

Returns:

A DataArray with dims (“feature”, “locus”) and coords “locus”, “feature” (full paths), “feature_name” (basenames), and “source”.

Return type:

xarray.DataArray

Notes

All selected features must share a compatible numeric dtype.

mutopia.gtensor.gtensor.fetch_interactions(dataset, component_name)

Retrieve interaction effects for a specific component.

This function extracts the interaction matrix for a specified component, showing how the mutational spectrum varies across different genomic contexts (e.g., strand orientation, replication timing, gene regions).

Parameters:

• dataset (GTensorDataset) – Dataset containing component interaction data

• component_name (Union[str, int]) – Name or index of the component to retrieve

Returns:

DataArray containing the component’s interaction effects with appropriate dimensions and coordinates

Return type:

xr.DataArray

Raises:

ValueError – If the specified component is not found in the dataset

mutopia.gtensor.gtensor.fetch_shared_effects(dataset, component_name)

Retrieve shared effects for a specific component.

This function extracts the shared effects matrix for a specified component, representing effects that are common across different contexts or conditions. Shared effects capture baseline mutational patterns that don’t vary with genomic features.

Parameters:

• dataset (GTensorDataset) – Dataset containing component shared effects data

• component_name (Union[str, int]) – Name or index of the component to retrieve

Returns:

DataArray containing the component’s shared effects with appropriate dimensions and coordinates

Return type:

xr.DataArray

Raises:

ValueError – If the specified component is not found in the dataset

mutopia.gtensor.gtensor.fetch_source(dataset, source)

Extract and restructure data for a specific source from a multi-source dataset.

This function filters and reorganizes a dataset to contain only data relevant to a specified source, while maintaining shared features and state variables that are common across all sources.

Parameters:

• dataset (GTensorDataset) – The input dataset containing data from multiple sources, organized with hierarchical variable names (e.g., “Features/source/variable”, “State/source/variable”).

• source (str) – The name of the source to extract data for. Must be present in the dataset.

Returns:

A new dataset containing: - Source-specific features and state variables (with paths flattened) - Shared features and state variables (common to all sources) - Other data variables from the original dataset - Updated name attribute reflecting the source - Source dimension removed if present

Return type:

GTensorDataset

Raises:

ValueError – If the specified source is not found in the dataset.

Notes

The function performs the following transformations: 1. Validates that the source exists in the dataset 2. Separates source-specific and shared variables from Features and State groups 3. Creates a rename mapping to flatten source-specific variable paths 4. Combines source-specific, shared, and other variables into a new dataset 5. Updates dataset attributes and coordinates while removing source dimension

mutopia.gtensor.gtensor.get_explanation(dataset, component)

Generate SHAP explanations for a specific model component.

This function extracts and formats SHAP values for interpretability analysis, creating an explanation object that can be used with SHAP visualization tools.

Parameters:

• dataset (GTensorDataset) – Dataset containing SHAP values and feature information

• component (str) – Name of the model component to explain

Returns:

SHAP explanation object with values, features, and display data

Return type:

shap.Explanation

Raises:

• ImportError – If SHAP library is not installed

• ValueError – If the specified component doesn’t have SHAP values in the dataset

mutopia.gtensor.gtensor.get_shap_summary(data, source=None)

Generate a summary of SHAP values for model components.

This function computes summary statistics for SHAP values across all components, including effect sizes (97th percentile of absolute SHAP values) and correlations between SHAP values and feature values. This provides a high-level view of which features have the strongest associations with each component.

Parameters:

• data (GTensorDataset) – Dataset containing SHAP values and feature information

• source (str, optional) – Source identifier to analyze. Required if the dataset is a mixture dataset with multiple sources.

Returns:

DataFrame with columns: - component: Component name - feature: Feature name - effect_size: 97th percentile of absolute SHAP values - correlation: Pearson correlation between SHAP values and feature values

Return type:

pd.DataFrame

Raises:

ValueError – If the dataset is a mixture dataset and no source is specified

mutopia.gtensor.gtensor.infer_source_celltypes(dataset)

Infer source cell types from feature names and assign to dataset coordinates.

This function examines the feature names in the dataset to identify unique source cell types based on directory structure. It then assigns these inferred cell types to the ‘source’ coordinate of the dataset.

Parameters:

dataset (GTensorDataset) – Input dataset containing features with potential source information

Returns:

Dataset with ‘source’ coordinate added, reflecting inferred cell types

Return type:

GTensorDataset

Raises:

ValueError – If no features are found in the dataset to infer sources from

mutopia.gtensor.gtensor.is_mixture_dataset(dataset)

Check if a dataset contains data from multiple sources.

This function determines whether the dataset is a mixture dataset by checking if it contains more than one source. Mixture datasets have source-specific features and require special handling for analysis.

Parameters:

dataset (GTensorDataset) – Input dataset to check

Returns:

True if the dataset contains multiple sources, False otherwise

Return type:

bool

mutopia.gtensor.gtensor.lazy_load(dataset)

Load a dataset lazily without samples or state information.

This is a convenience function that loads a dataset with minimal memory footprint by excluding sample data and state information.

Parameters:

dataset (str or path-like) – Path or identifier for the dataset to load

Returns:

Lazy dataset interface that loads data on demand

Return type:

GTensorDataset

mutopia.gtensor.gtensor.lazy_train_test_load(dataset, *test_chroms)

Load a dataset and perform lazy train/test split by chromosomes.

This convenience function combines lazy loading with train/test splitting, providing memory-efficient access to training and testing data.

Parameters:

• dataset (str or path-like) – Path or identifier for the dataset to load

• *test_chroms (str) – Chromosome names to reserve for the test set

Returns:

Training and testing dataset slicers

Return type:

tuple[LazySlicer, LazySlicer]

mutopia.gtensor.gtensor.list_components(dataset)

List all component names in the dataset.

This function extracts and returns the names of all model components (mutational signatures or processes) present in the dataset.

Parameters:

dataset (GTensorDataset) – Input dataset containing model components

Returns:

List of component names

Return type:

List[str]

Raises:

ValueError – If the dataset does not contain a ‘component’ coordinate

mutopia.gtensor.gtensor.list_sources(dataset)

List all source identifiers in the dataset.

This function extracts and returns the names of all sources present in the dataset. Sources typically represent different cell types, tissues, or experimental conditions.

Parameters:

dataset (GTensorDataset) – Input dataset containing source information

Returns:

List of source names. Returns an empty list if the dataset has no ‘source’ coordinate defined.

Return type:

List[str]

mutopia.gtensor.gtensor.load_dataset(dataset, with_samples=True, with_state=True)

Load a dataset from disk with configurable loading options.

This function loads a dataset from disk storage. The loading behavior can be customized based on whether samples and state information should be included.

Parameters:

• dataset (str or path-like) – Path or identifier for the dataset to load

• with_samples (bool, default=True) – Whether to load sample data along with the dataset structure

• with_state (bool, default=True) – Whether to load state information (model parameters, etc.)

Returns:

Loaded dataset interface. Returns LazySampleLoader if with_samples=False, otherwise returns CorpusInterface

Return type:

GTensorDataset

mutopia.gtensor.gtensor.make_mixture_dataset(**datasets)

Create a mixed dataset by combining multiple source datasets.

This function merges multiple datasets, renaming their features and state variables to include source identifiers, enabling comparative analysis across different data sources.

Parameters:

**datasets (GTensorDataset) – Named datasets to combine. Keys become source identifiers.

Returns:

Combined dataset with source-specific feature namespaces

Return type:

GTensorDataset

mutopia.gtensor.gtensor.match_dims(X, **dim_sizes)

mutopia.gtensor.gtensor.mutate_method(func)

Decorator function to modify a dataset in place for class methods.

This decorator allows running mutations on a dataset without disrupting the interface chains, specifically for methods that take ‘self’ as the first parameter.

Parameters:

func (callable) – Method that takes self and dataset as first two arguments and returns a modified dataset

Returns:

Wrapped method that can be used with dataset.mutate()

Return type:

callable

mutopia.gtensor.gtensor.num_sources(dataset)

Get the number of distinct sources in a dataset.

This function counts the number of unique sources present in the dataset’s ‘source’ coordinate, which is useful for determining if the dataset contains data from multiple cell types or conditions.

Parameters:

dataset (GTensorDataset) – Input dataset to query for sources

Returns:

Number of distinct sources in the dataset. Returns 0 if no sources are defined.

Return type:

int

mutopia.gtensor.gtensor.rename_components(dataset, names)

Rename the components of the model and update the dataset coordinates accordingly.

Parameters:

• dataset (GTensorDataset) – The dataset containing model components to be renamed.

• names (List[str]) – New names for the components. Must have the same length as the number of components in the model.

Returns:

The dataset with updated component names in coordinates.

Return type:

GTensorDataset

Raises:

• ValueError – If the number of provided names doesn’t match the number of components.

• KeyError – If some components in the dataset’s “shap_component” coordinate don’t match the model components.

Notes

This method also updates the internal _component_names attribute of the model.

mutopia.gtensor.gtensor.slice_regions(dataset, *regions, lazy=False)

Extract genomic regions that overlap with specified intervals.

This function filters the dataset to include only regions that overlap with any of the specified genomic intervals. Intervals can be specified in multiple formats: “chr:start-end”, “chr” (entire chromosome), or a comma-separated list of such specifications.

Parameters:

• dataset (GTensorDataset) – Input dataset containing genomic regions

• regions (str) – Region specification(s) in formats: - “chr:start-end” (e.g., “chr1:1000-2000”) - “chr” (entire chromosome, e.g., “chr1”) - List of any of the above

• lazy (bool, default=False) – Whether to return a lazy slicer instead of materializing the data

Returns:

Filtered dataset containing only overlapping regions

Return type:

GTensorDataset

Raises:

ValueError – If no regions match the specified query intervals

mutopia.gtensor.gtensor.slice_samples(dataset, samples)

Extract a subset of samples from the dataset.

This function filters the dataset to include only the specified samples, enabling focused analysis on particular samples of interest while maintaining all other dataset dimensions and attributes.

Parameters:

• dataset (GTensorDataset) – Input dataset containing multiple samples

• samples (List[str]) – List of sample names to extract from the dataset. Sample names must exist in the dataset’s sample coordinate.

Returns:

Filtered dataset containing only the specified samples wrapped in a SampleSlice interface

Return type:

GTensorDataset

Raises:

KeyError – If any of the specified samples are not found in the dataset

Notes

If an empty list is provided, the original dataset is returned unchanged. The function uses the mutate pattern to maintain interface chain compatibility.

mutopia.gtensor.gtensor.train_test_split(dataset, *test_chroms, lazy=False)

Split a dataset into training and testing sets based on chromosomes.

This function splits the dataset by chromosomes, with specified chromosomes reserved for testing and the remainder used for training. The split can be performed eagerly (loading all data) or lazily (for memory efficiency).

Parameters:

• dataset (GTensorDataset) – Input dataset to split

• *test_chroms (Union[str, List[str]]) – Chromosome names to reserve for the test set. Can be provided as multiple string arguments or lists of strings

• lazy (bool, default=False) – Whether to perform lazy splitting. If True, returns LazySlicer objects that don’t load data until accessed

Returns:

Training and testing dataset interfaces

Return type:

tuple[GTensorDataset, GTensorDataset]

Raises:

ValueError – If no test chromosomes are provided or none of the specified chromosomes are found in the dataset

mutopia.gtensor.gtensor.unstack_regions(dataset)

Unstack regions from a compressed format to full coordinate arrays.

This function expands region data from a compact representation to full coordinate arrays, using external region file information to reconstruct chromosome, start, and end coordinates.

Parameters:

dataset (GTensorDataset) – Dataset with stacked region representation

Returns:

Dataset with unstacked region coordinates

Return type:

GTensorDataset