Issue with Pathway Detection and PEM Calculation in CytoTalk

[Sophia409]

Dear CytoTalk Developers,

I hope this message finds you well. I'm writing to report an issue I've encountered while using the CytoTalk::run_cytotalk() function, and I would greatly appreciate your expertise in resolving it.

Issue Description:

Consistently getting "No pathways found, analysis skipped!" message across multiple datasets, including:

My own single-cell RNA-seq data

The built-in example data (CytoTalk::scrna_cyto)

The problem persists regardless of whether I use different species parameters (pcg_mouse/pcg_human, lrp_mouse/lrp_human)

Additional observation:

The PEM matrix (result$pem) contains numerous NaN and Inf values

This occurs even with the example dataset where I'd expect valid calculations

Could you please advise:

What might be causing these NaN/Inf values in the PEM matrix?

Why pathway detection consistently fails despite using the example data?

Any recommended troubleshooting steps or potential workarounds?

I've attached the output of sessionInfo() and sample PEM matrix summaries for reference. Thank you very much for your time and for developing this valuable tool. I'd be happy to provide any additional information that might help diagnose this issue.

Best regards,

Sophia

#自己的数据
> lst_scrna$mat <- sce_filt_ImmN[["RNA"]]$data
> lst_scrna$cell_types <- sce_filt_ImmN@meta.data$cellclass2
> # set required parameters
> type_a <- "GLU"
> type_b <- "AS"
> # run CytoTalk process
> results <- CytoTalk::run_cytotalk(lst_scrna, type_a, type_b)
[1 / 8] (22:02:07) Preprocessing...
[2 / 8] (22:05:24) Mutual information matrix...
[3 / 8] (22:07:55) Indirect edge-filtered network...
[4 / 8] (22:08:20) Integrate network...
[5 / 8] (22:09:25) PCSF...
[6 / 8] (22:09:31) Determine best signaling network...
[7 / 8] (22:09:31) Generate network output...
[8 / 8] (22:09:32) NOTE: No pathways found, analysis skipped!
#包自带的数据
>   result <- CytoTalk::run_cytotalk(CytoTalk::scrna_cyto,
+                                    cell_type_a, cell_type_b,
+                                    cutoff_a, cutoff_b,
+                                    cores = 20,pcg = CytoTalk::pcg_mouse,lrp = CytoTalk::lrp_mouse,dir_out = './CytoTalk')
[1 / 8] (13:41:48) Preprocessing...
[2 / 8] (13:41:55) Mutual information matrix...
[3 / 8] (13:42:24) Indirect edge-filtered network...
[4 / 8] (13:42:28) Integrate network...
[5 / 8] (13:42:43) PCSF...
[6 / 8] (13:42:53) Determine best signaling network...
[7 / 8] (13:42:54) Generate network output...
[8 / 8] (13:42:55) NOTE: No pathways found, analysis skipped!
> unique(scrna_cyto$cell_types)
[1] "EndothelialCells"       "LuminalEpithelialCells" "Macrophages"           
>   result <- CytoTalk::run_cytotalk(CytoTalk::scrna_cyto,
+                                    cell_type_a, cell_type_b,
+                                    cutoff_a, cutoff_b,
+                                    cores = 20)
[1 / 8] (13:43:49) Preprocessing...
[2 / 8] (13:43:54) Mutual information matrix...
[3 / 8] (13:44:21) Indirect edge-filtered network...
[4 / 8] (13:44:25) Integrate network...
[5 / 8] (13:44:35) PCSF...
[6 / 8] (13:44:37) Determine best signaling network...
[7 / 8] (13:44:38) Generate network output...
[8 / 8] (13:44:38) NOTE: No pathways found, analysis skipped!
>   str(result)
List of 5
 $ params        :List of 9
  ..$ cell_type_a: chr "Macrophages"
  ..$ cell_type_b: chr "LuminalEpithelialCells"
  ..$ cutoff_a   : num 0.6
  ..$ cutoff_b   : num 0.6
  ..$ beta_max   : num 100
  ..$ omega_min  : num 0.5
  ..$ omega_max  : num 0.5
  ..$ depth      : num 3
  ..$ ntrial     : num 1000
 $ pem           : num [1:23341, 1:3] NaN NaN 0.468 -0.117 -0.281 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:23341] "Xkr4" "Rp1" "Sox17" "Mrpl15" ...
  .. ..$ : chr [1:3] "EndothelialCells" "LuminalEpithelialCells" "Macrophages"
 $ integrated_net:List of 2
  ..$ nodes:'data.frame':	2367 obs. of  4 variables:
  .. ..$ node          : chr [1:2367] "Tcea1__Macrophages" "Tram1__Macrophages" "Rpl7__Macrophages" "Ptpn18__Macrophages" ...
  .. ..$ prize         : num [1:2367] 0.001582 0 0.000421 0.012011 0 ...
  .. ..$ pem           : num [1:2367] 0.0497 0 0.0112 0.3409 0 ...
  .. ..$ gene_relevance: num [1:2367] 0.0318 0.0379 0.0375 0.0352 0.0333 ...
  ..$ edges:'data.frame':	41784 obs. of  3 variables:
  .. ..$ node1: chr [1:41784] "Tram1__Macrophages" "Stk17b__Macrophages" "Arl4c__Macrophages" "Tpr__Macrophages" ...
  .. ..$ node2: chr [1:41784] "Tcea1__Macrophages" "Tcea1__Macrophages" "Tcea1__Macrophages" "Tcea1__Macrophages" ...
  .. ..$ cost : num [1:41784] 0.569 0.643 0.707 0.798 0.798 ...
 $ pcst          :List of 3
  ..$ occurances   :List of 2
  .. ..$ nodes:'data.frame':	27714 obs. of  3 variables:
  .. .. ..$ beta : int [1:27714] 1 2 3 4 5 6 7 8 8 8 ...
  .. .. ..$ omega: num [1:27714] 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 ...
  .. .. ..$ node : chr [1:27714] "ARTI" "ARTI" "ARTI" "ARTI" ...
  .. ..$ edges:'data.frame':	27614 obs. of  4 variables:
  .. .. ..$ beta : int [1:27614] 8 8 8 8 8 9 9 9 9 9 ...
  .. .. ..$ omega: num [1:27614] 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 ...
  .. .. ..$ node1: chr [1:27614] "ARTI" "ARTI" "ARTI" "ARTI" ...
  .. .. ..$ node2: chr [1:27614] "Ly86__Macrophages" "Ly86__Macrophages" "Ly86__Macrophages" "Ly86__Macrophages" ...
  ..$ ks_test_pval :'data.frame':	91 obs. of  3 variables:
  .. ..$ beta : num [1:91] 100 60 61 62 63 64 65 66 67 68 ...
  .. ..$ omega: num [1:91] 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 ...
  .. ..$ pval : num [1:91] 0 0 0 0 0 0 0 0 0 0 ...
  ..$ final_network:'data.frame':	626 obs. of  10 variables:
  .. ..$ node1      : chr [1:626] "Rpl4" "Rpl4" "Rpl4" "Rpl4" ...
  .. ..$ node2      : chr [1:626] "Rps11" "Rps11" "Rps11" "Rps11" ...
  .. ..$ node1_type : chr [1:626] "Macrophages" "Macrophages" "Macrophages" "Macrophages" ...
  .. ..$ node2_type : chr [1:626] "Macrophages" "Macrophages" "Macrophages" "Macrophages" ...
  .. ..$ node1_prize: num [1:626] 0.00243 0.00243 0.00243 0.00243 0.00243 ...
  .. ..$ node2_prize: num [1:626] 0.00524 0.00524 0.00524 0.00524 0.00524 ...
  .. ..$ node1_pem  : num [1:626] 0.0598 0.0598 0.0598 0.0598 0.0598 ...
  .. ..$ node2_pem  : num [1:626] 0.139 0.139 0.139 0.139 0.139 ...
  .. ..$ is_ct_edge : logi [1:626] FALSE FALSE FALSE FALSE FALSE FALSE ...
  .. ..$ cost       : num [1:626] 0 0 0 0 0 0 0 0 0 0 ...
 $ pathways      : NULL
>   # 检查PEM质量
>   summary(results$pem)  # 查看NaN/Inf比例
Error: object 'results' not found
>   # 检查PEM质量
>   summary(result$pem)  # 查看NaN/Inf比例
 EndothelialCells LuminalEpithelialCells  Macrophages    
 Min.   :  -Inf   Min.   :  -Inf         Min.   :  -Inf  
 1st Qu.:-1.383   1st Qu.:-0.332         1st Qu.:-0.948  
 Median :-0.100   Median : 0.022         Median :-0.135  
 Mean   :  -Inf   Mean   :  -Inf         Mean   :  -Inf  
 3rd Qu.: 0.158   3rd Qu.: 0.219         3rd Qu.: 0.094  
 Max.   : 0.477   Max.   : 0.477         Max.   : 0.477  
 NA's   :10213    NA's   :10213          NA's   :10213 
> sessionInfo()
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22000)

Matrix products: default


locale:
[1] LC_COLLATE=Chinese (Simplified)_China.utf8  LC_CTYPE=Chinese (Simplified)_China.utf8    LC_MONETARY=Chinese (Simplified)_China.utf8
[4] LC_NUMERIC=C                                LC_TIME=Chinese (Simplified)_China.utf8    

time zone: Asia/Shanghai
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] CytoTalk_0.99.9

loaded via a namespace (and not attached):
  [1] IRanges_2.40.0              R.methodsS3_1.8.2           vroom_1.6.5                 progress_1.2.3             
  [5] parmigene_1.1.1             goftest_1.2-3               Biostrings_2.74.0           HDF5Array_1.34.0           
  [9] vctrs_0.6.5                 ggtangle_0.0.6              spatstat.random_3.3-2       digest_0.6.36              
 [13] png_0.1-8                   corpcor_1.6.10              shape_1.4.6.1               slingshot_2.14.0           
 [17] ggrepel_0.9.6               deldir_2.0-4                parallelly_1.43.0           MASS_7.3-61                
 [21] Signac_1.14.0               reshape2_1.4.4              httpuv_1.6.15               foreach_1.5.2              
 [25] BiocGenerics_0.52.0         qvalue_2.38.0               withr_3.0.2                 xfun_0.51                  
 [29] ggfun_0.1.8                 survival_3.7-0              memoise_2.0.1               proxyC_0.4.1               
 [33] clusterProfiler_4.14.6      MatrixModels_0.5-4          gson_0.1.0                  princurve_2.1.6            
 [37] SCP_0.5.6                   tidytree_0.4.6              zoo_1.8-12                  GlobalOptions_0.1.2        
 [41] pbapply_1.7-2               R.oo_1.27.0                 prettyunits_1.2.0           KEGGREST_1.46.0            
 [45] promises_1.3.2              evmix_2.12                  httr_1.4.7                  globals_0.17.0             
 [49] fitdistrplus_1.2-2          rhdf5filters_1.18.0         rhdf5_2.50.0                rstudioapi_0.17.1          
 [53] UCSC.utils_1.2.0            miniUI_0.1.1.1              generics_0.1.3              DOSE_4.0.0                 
 [57] ggalluvial_0.12.5           curl_6.0.1                  S4Vectors_0.44.0            zlibbioc_1.52.0            
 [61] ggraph_2.2.1                polyclip_1.10-7             GenomeInfoDbData_1.2.13     SparseArray_1.6.0          
 [65] xtable_1.8-4                stringr_1.5.1               doParallel_1.0.17           evaluate_1.0.3             
 [69] S4Arrays_1.6.0              BiocFileCache_2.14.0        infotheo_1.2.0.1            hms_1.1.3                  
 [73] GenomicRanges_1.58.0        irlba_2.3.5.1               colorspace_2.1-0            filelock_1.0.3             
 [77] ROCR_1.0-11                 reticulate_1.40.0           spatstat.data_3.1-6         magrittr_2.0.3             
 [81] lmtest_0.9-40               later_1.4.1                 viridis_0.6.5               ggtree_3.14.0              
 [85] lattice_0.22-6              spatstat.geom_3.3-4         future.apply_1.11.3         SparseM_1.84-2             
 [89] scattermore_1.2             cowplot_1.1.3               matrixStats_1.4.1           RcppAnnoy_0.0.22           
 [93] pillar_1.10.2               nlme_3.1-165                iterators_1.0.14            compiler_4.4.1             
 [97] RSpectra_0.16-2             stringi_1.8.4               tensor_1.5                  SummarizedExperiment_1.36.0
[101] plyr_1.8.9                  crayon_1.5.3                abind_1.4-8                 ggdendro_0.2.0             
[105] gridGraphics_0.5-1          sp_2.1-4                    graphlayouts_1.2.1          bit_4.5.0.1                
[109] dplyr_1.1.4                 fastmatch_1.1-4             codetools_0.2-20            GetoptLong_1.0.5           
[113] plotly_4.10.4               mime_0.12                   splines_4.4.1               circlize_0.4.16            
[117] Rcpp_1.0.12                 fastDummies_1.7.5           quantreg_5.99.1             dbplyr_2.5.0               
[121] gridtext_0.1.5              knitr_1.50                  blob_1.2.4                  here_1.0.1                 
[125] clue_0.3-66                 scRepertoire_2.2.1          fs_1.6.4                    listenv_0.9.1              
[129] evd_2.3-7.1                 openxlsx_4.2.8              gsl_2.1-8                   ggplotify_0.1.2            
[133] tibble_3.2.1                Matrix_1.7-1                tzdb_0.4.0                  tweenr_2.0.3               
[137] pkgconfig_2.0.3             pheatmap_1.0.12             tools_4.4.1                 cachem_1.1.0               
[141] R.cache_0.16.0              RSQLite_2.3.9               viridisLite_0.4.2           DBI_1.2.3                  
[145] fastmap_1.2.0               rmarkdown_2.29              scales_1.3.0                grid_4.4.1                 
[149] ica_1.0-3                   Seurat_5.1.0                Rsamtools_2.22.0            patchwork_1.3.0            
[153] BiocManager_1.30.25         dotCall64_1.2               RANN_2.6.2                  farver_2.1.2               
[157] tidygraph_1.3.1             yaml_2.3.10                 VGAM_1.1-12                 MatrixGenerics_1.18.1      
[161] cli_3.6.3                   purrr_1.0.2                 stats4_4.4.1                leiden_0.4.3.1             
[165] lifecycle_1.0.4             uwot_0.2.2                  Biobase_2.66.0              BiocParallel_1.40.0        
[169] gtable_0.3.6                rjson_0.2.23                ggridges_0.5.6              progressr_0.15.1           
[173] cubature_2.1.1              parallel_4.4.1              ape_5.8                     jsonlite_1.8.8             
[177] RcppHNSW_0.6.0              bitops_1.0-9                ggplot2_3.5.1               bit64_4.6.0-1              
[181] assertthat_0.2.1            Rtsne_0.17                  yulab.utils_0.2.0           spatstat.utils_3.1-3       
[185] SeuratObject_5.0.2          zip_2.3.1                   highr_0.11                  GOSemSim_2.32.0            
[189] spatstat.univar_3.1-1       R.utils_2.13.0              truncdist_1.0-2             lazyeval_0.2.2             
[193] shiny_1.10.0                htmltools_0.5.8.1           enrichplot_1.26.2           GO.db_3.20.0               
[197] iNEXT_3.0.1                 sctransform_0.4.1           rappdirs_0.3.3              scplotter_0.1.2            
[201] glue_1.7.0                  spam_2.11-0                 httr2_1.1.2                 XVector_0.46.0             
[205] RCurl_1.98-1.16             rprojroot_2.0.4             treeio_1.30.0               gridExtra_2.3              
[209] igraph_2.1.2                TrajectoryUtils_1.14.0      plotthis_0.6.1              R6_2.5.1                   
[213] tidyr_1.3.1                 SingleCellExperiment_1.28.1 forcats_1.0.0               RcppRoll_0.3.1             
[217] cluster_2.1.6               Rhdf5lib_1.28.0             stringdist_0.9.14           aplot_0.2.5                
[221] GenomeInfoDb_1.42.1         DelayedArray_0.32.0         tidyselect_1.2.1            ggforce_0.4.2              
[225] xml2_1.3.6                  AnnotationDbi_1.68.0        future_1.40.0               munsell_0.5.1              
[229] KernSmooth_2.23-24          data.table_1.16.2           htmlwidgets_1.6.4           fgsea_1.32.0               
[233] ComplexHeatmap_2.22.0       RColorBrewer_1.1-3          biomaRt_2.62.0              rlang_1.1.4                
[237] spatstat.sparse_3.1-0       spatstat.explore_3.3-3      ggnewscale_0.5.1  

[huBioinfo]

Hi, please see the two potential causes below.

(1) CytoTalk employs a python module "pcst_fast" to conduct the PCSF search and this module has some compatibility issue with numpy>=2.0 (In fact, we just recently discovered this issue). Please check if the NumPy version in your Python environment is 2.0 or higher. If so, this might be the issue that you cannot identify correct pcsf. The solution is to downgrade NumPy to a 1.x version. Our tested version, ​​1.26.4​​, works well.
(2) If you still got few pathways, you might want to adjust the hyperparameters "cutoff_a" and "cutoff_b", which are used for filtering lowly-expressed genes.

Also, in CytoTalk, it's normal for the PEM matrix to contain NaN and -Inf values and these do not affect final results. Specifically, NaN values occur when a gene exhibits no expression across all cells, causing the PEM calculation to return NaN. Conversely, -Inf values occur when a gene is unexpressed in one cell type but expressed in others, leading to -Inf. Note that both types of values will be filtered out in downstream processing steps, ensuring they have no impact on the final analysis.

Please let us know if the issue still exists. Thanks.

[Sophia409]

Dear CytoTalk Team, @huBioinfo

Thank you very much for your prompt and helpful response.

I followed your recommendation and successfully downgraded NumPy to version 1.26.4 in the relevant Python environment.

However, the run_cytotalk function now encounters a new error during Step 7: "Generate network output". The specific error is:
Error: Edge length [XXXXX] larger than maximum 65535 allowed. Check for overwide node(s).

I have tried adjusting various parameters including cutoff_a, cutoff_b, and ntrial in an attempt to resolve this, but the error persists.

I've attached the output of sessionInfo() and code for reference. Could you please provide any advice or suggestions on how to overcome this new obstacle? Any guidance would be greatly appreciated.

Thank you again for your time and support.

Best regards,

> library(reticulate)
> py_config()
python:         D:/software/Miniconda3/envs/cytotalk/python.exe
libpython:      D:/software/Miniconda3/envs/cytotalk/python39.dll
pythonhome:     D:/software/Miniconda3/envs/cytotalk
version:        3.9.23 | packaged by conda-forge | (main, Jun  4 2025, 17:49:16) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          D:/software/Miniconda3/envs/cytotalk/Lib/site-packages/numpy
numpy_version:  1.26.4

NOTE: Python version was forced by use_python() function
> summary(lst_scrna)
           Length   Class     Mode     
mat        23587200 dgCMatrix S4       
cell_types      700 -none-    character
> str(lst_scrna)
List of 2
 $ mat       :Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  .. ..@ i       : int [1:934451] 25 35 43 53 63 114 115 131 147 152 ...
  .. ..@ p       : int [1:701] 0 862 1833 3357 5986 6872 8038 9072 11023 12170 ...
  .. ..@ Dim     : int [1:2] 33696 700
  .. ..@ Dimnames:List of 2
  .. .. ..$ : chr [1:33696] "Xkr4" "Gm1992" "Gm19938" "Gm37381" ...
  .. .. ..$ : chr [1:700] "AACTTGCACC_AAGACCTAGG_1_1_1" "GTGACATACA_AAGATCAATT_1_1_1" "CTTACCGGTG_AAGATTAACC_1_1_1" "CCGATGAGCA_AATAATGCAC_1_1_1" ...
  .. ..@ x       : num [1:934451] 3 1 1 1 1 2 1 5 1 1 ...
  .. ..@ factors : list()
 $ cell_types: chr [1:700] "OPC" "OPC" "GLU" "GABA" ...
> table(lst_scrna$cell_type)

  AS GABA  GLU ImmN   MG   OD  OPC 
 100  100  100  100  100  100  100 
> # run CytoTalk process
> results <- CytoTalk::run_cytotalk(lst_scrna, type_a, type_b,dir_out = './CytoTalk2',cores = 10,pcg = CytoTalk::pcg_mouse,lrp = CytoTalk::lrp_mouse,cutoff_a = 0.4,cutoff_b = 0.4,ntrial = 500)
[1 / 8] (16:42:48) Preprocessing...
[2 / 8] (16:42:58) Mutual information matrix...
[3 / 8] (16:43:16) Indirect edge-filtered network...
[4 / 8] (16:43:18) Integrate network...
[5 / 8] (16:43:25) PCSF...
[6 / 8] (16:43:31) Determine best signaling network...
[7 / 8] (16:43:33) Generate network output...
Error: Error: Edge length 86655.500000 larger than maximum 65535 allowed.
Check for overwide node(s).
> # run CytoTalk process
> results <- CytoTalk::run_cytotalk(lst_scrna, type_a, type_b,dir_out = './CytoTalk2',cores = 10,pcg = CytoTalk::pcg_mouse,lrp = CytoTalk::lrp_mouse,cutoff_a = 0.4,cutoff_b = 0.4,ntrial = 500,omega_max = 1.5)
[1 / 8] (16:56:21) Preprocessing...
[2 / 8] (16:56:30) Mutual information matrix...
[3 / 8] (16:56:52) Indirect edge-filtered network...
[4 / 8] (16:56:53) Integrate network...
[5 / 8] (16:57:01) PCSF...
[6 / 8] (17:14:17) Determine best signaling network...
[7 / 8] (17:18:22) Generate network output...
Error: Error: Edge length 86655.500000 larger than maximum 65535 allowed.
Check for overwide node(s).
> results
Error: object 'results' not found
> # run CytoTalk process
> results <- CytoTalk::run_cytotalk(lst_scrna, type_a, type_b,dir_out = './CytoTalk2',cores = 10,pcg = CytoTalk::pcg_mouse,lrp = CytoTalk::lrp_mouse,cutoff_a = 0.4,cutoff_b = 0.4,ntrial = 50)
[1 / 8] (18:44:15) Preprocessing...
[2 / 8] (18:44:25) Mutual information matrix...
[3 / 8] (18:44:44) Indirect edge-filtered network...
[4 / 8] (18:44:46) Integrate network...
[5 / 8] (18:44:53) PCSF...
[6 / 8] (18:45:03) Determine best signaling network...
[7 / 8] (18:45:05) Generate network output...
Error: Error: Edge length 86655.500000 larger than maximum 65535 allowed.
Check for overwide node(s).
> library(reticulate)
> py_config()
python:         D:/software/Miniconda3/envs/cytotalk/python.exe
libpython:      D:/software/Miniconda3/envs/cytotalk/python39.dll
pythonhome:     D:/software/Miniconda3/envs/cytotalk
version:        3.9.23 | packaged by conda-forge | (main, Jun  4 2025, 17:49:16) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          D:/software/Miniconda3/envs/cytotalk/Lib/site-packages/numpy
numpy_version:  1.26.4

NOTE: Python version was forced by use_python() function