2025-06-17 12:43:01,839::INFO::main: Starting jointly-hic 2025-06-17 12:43:12,299::INFO::__init__: PostProcessor :: configuration: ==================== Post Processor Configuration ==================== parquet_file : output_2025_06_12_13_36_PCA-32_50000bp_hg38_embeddings.pq output : output_6_12_2025_post_process umap_neighbours : [30] kmeans_clusters : [5, 6, 8, 10] leiden_resolutions : [0.1, 0.2, 0.3, 0.5, 0.8, 1.0] method : PCA log_level : INFO ======================================================================== 2025-06-17 12:43:12,299::INFO::__init__: Loading embeddings from parquet file: output_2025_06_12_13_36_PCA-32_50000bp_hg38_embeddings.pq 2025-06-17 12:43:14,539::INFO::run: Running post-processing 2025-06-17 12:43:21,266::INFO::normalize_embeddings: Normalizing embeddings for DE-FA-DSG-DdeI-DpnII-P1P2.hg38.mapq_30.1000.mcool 2025-06-17 12:43:29,852::INFO::normalize_embeddings: Normalizing embeddings for DE_pool.hg38.mapq_30.100.mcool 2025-06-17 12:43:43,906::INFO::normalize_embeddings: Normalizing embeddings for ENCFF004YZQ.mcool 2025-06-17 12:43:49,063::INFO::normalize_embeddings: Normalizing embeddings for ENCFF009ONH.mcool 2025-06-17 12:43:53,250::INFO::normalize_embeddings: Normalizing embeddings for ENCFF019YVI.mcool 2025-06-17 12:43:56,198::INFO::normalize_embeddings: Normalizing embeddings for ENCFF029UAL.mcool 2025-06-17 12:43:59,101::INFO::normalize_embeddings: Normalizing embeddings for ENCFF035BLF.mcool 2025-06-17 12:44:02,072::INFO::normalize_embeddings: Normalizing embeddings for ENCFF044TCQ.mcool 2025-06-17 12:44:05,004::INFO::normalize_embeddings: Normalizing embeddings for ENCFF059WMS.mcool 2025-06-17 12:44:07,813::INFO::normalize_embeddings: Normalizing embeddings for ENCFF076LWH.mcool 2025-06-17 12:44:10,650::INFO::normalize_embeddings: Normalizing embeddings for ENCFF081PMO.mcool 2025-06-17 12:44:13,496::INFO::normalize_embeddings: Normalizing embeddings for ENCFF094KKX.mcool 2025-06-17 12:44:16,332::INFO::normalize_embeddings: Normalizing embeddings for ENCFF136XCV.mcool 2025-06-17 12:44:19,181::INFO::normalize_embeddings: Normalizing embeddings for ENCFF140ROU.mcool 2025-06-17 12:44:22,028::INFO::normalize_embeddings: Normalizing embeddings for ENCFF149IHN.mcool 2025-06-17 12:44:24,867::INFO::normalize_embeddings: Normalizing embeddings for ENCFF176LIZ.mcool 2025-06-17 12:44:27,736::INFO::normalize_embeddings: Normalizing embeddings for ENCFF185AYZ.mcool 2025-06-17 12:44:30,631::INFO::normalize_embeddings: Normalizing embeddings for ENCFF193CQL.mcool 2025-06-17 12:44:33,536::INFO::normalize_embeddings: Normalizing embeddings for ENCFF197OWW.mcool 2025-06-17 12:44:37,868::INFO::normalize_embeddings: Normalizing embeddings for ENCFF224HKR.mcool 2025-06-17 12:44:42,586::INFO::normalize_embeddings: Normalizing embeddings for ENCFF235LCO.mcool 2025-06-17 12:44:45,850::INFO::normalize_embeddings: Normalizing embeddings for ENCFF251VFA.mcool 2025-06-17 12:44:48,917::INFO::normalize_embeddings: Normalizing embeddings for ENCFF252ANV.mcool 2025-06-17 12:44:52,411::INFO::normalize_embeddings: Normalizing embeddings for ENCFF273XBU.mcool 2025-06-17 12:44:56,036::INFO::normalize_embeddings: Normalizing embeddings for ENCFF284OEA.mcool 2025-06-17 12:44:59,861::INFO::normalize_embeddings: Normalizing embeddings for ENCFF309UNV.mcool 2025-06-17 12:45:03,593::INFO::normalize_embeddings: Normalizing embeddings for ENCFF333WCK.mcool 2025-06-17 12:45:07,410::INFO::normalize_embeddings: Normalizing embeddings for ENCFF355NFJ.mcool 2025-06-17 12:45:11,197::INFO::normalize_embeddings: Normalizing embeddings for ENCFF355VJW.mcool 2025-06-17 12:45:15,216::INFO::normalize_embeddings: Normalizing embeddings for ENCFF395INO.mcool 2025-06-17 12:45:19,526::INFO::normalize_embeddings: Normalizing embeddings for ENCFF417GBZ.mcool 2025-06-17 12:45:23,489::INFO::normalize_embeddings: Normalizing embeddings for ENCFF442IGJ.mcool 2025-06-17 12:45:27,273::INFO::normalize_embeddings: Normalizing embeddings for ENCFF458VBB.mcool 2025-06-17 12:45:31,151::INFO::normalize_embeddings: Normalizing embeddings for ENCFF493SFI.mcool 2025-06-17 12:45:35,116::INFO::normalize_embeddings: Normalizing embeddings for ENCFF493YNC.mcool 2025-06-17 12:45:39,017::INFO::normalize_embeddings: Normalizing embeddings for ENCFF496GEU.mcool 2025-06-17 12:45:42,950::INFO::normalize_embeddings: Normalizing embeddings for ENCFF499BVX.mcool 2025-06-17 12:45:46,727::INFO::normalize_embeddings: Normalizing embeddings for ENCFF515ZBF.mcool 2025-06-17 12:45:50,481::INFO::normalize_embeddings: Normalizing embeddings for ENCFF520GFL.mcool 2025-06-17 12:45:54,262::INFO::normalize_embeddings: Normalizing embeddings for ENCFF545GBW.mcool 2025-06-17 12:45:58,202::INFO::normalize_embeddings: Normalizing embeddings for ENCFF556RLR.mcool 2025-06-17 12:46:01,681::INFO::normalize_embeddings: Normalizing embeddings for ENCFF558DYU.mcool 2025-06-17 12:46:05,547::INFO::normalize_embeddings: Normalizing embeddings for ENCFF571GTF.mcool 2025-06-17 12:46:09,249::INFO::normalize_embeddings: Normalizing embeddings for ENCFF579CAR.mcool 2025-06-17 12:46:13,212::INFO::normalize_embeddings: Normalizing embeddings for ENCFF586MQY.mcool 2025-06-17 12:46:16,811::INFO::normalize_embeddings: Normalizing embeddings for ENCFF591MHA.mcool 2025-06-17 12:46:20,654::INFO::normalize_embeddings: Normalizing embeddings for ENCFF605CEN.mcool 2025-06-17 12:46:24,296::INFO::normalize_embeddings: Normalizing embeddings for ENCFF614IPZ.mcool 2025-06-17 12:46:27,972::INFO::normalize_embeddings: Normalizing embeddings for ENCFF625VNK.mcool 2025-06-17 12:46:31,577::INFO::normalize_embeddings: Normalizing embeddings for ENCFF628LQQ.mcool 2025-06-17 12:46:35,510::INFO::normalize_embeddings: Normalizing embeddings for ENCFF653HCO.mcool 2025-06-17 12:46:39,418::INFO::normalize_embeddings: Normalizing embeddings for ENCFF654YIQ.mcool 2025-06-17 12:46:43,222::INFO::normalize_embeddings: Normalizing embeddings for ENCFF660JWA.mcool 2025-06-17 12:46:47,073::INFO::normalize_embeddings: Normalizing embeddings for ENCFF663FRL.mcool 2025-06-17 12:46:50,802::INFO::normalize_embeddings: Normalizing embeddings for ENCFF684YCT.mcool 2025-06-17 12:46:54,727::INFO::normalize_embeddings: Normalizing embeddings for ENCFF687ATD.mcool 2025-06-17 12:46:58,662::INFO::normalize_embeddings: Normalizing embeddings for ENCFF700CYI.mcool 2025-06-17 12:47:02,654::INFO::normalize_embeddings: Normalizing embeddings for ENCFF705YZH.mcool 2025-06-17 12:47:06,242::INFO::normalize_embeddings: Normalizing embeddings for ENCFF711XSR.mcool 2025-06-17 12:47:09,393::INFO::normalize_embeddings: Normalizing embeddings for ENCFF736ITL.mcool 2025-06-17 12:47:13,033::INFO::normalize_embeddings: Normalizing embeddings for ENCFF782WVZ.mcool 2025-06-17 12:47:16,522::INFO::normalize_embeddings: Normalizing embeddings for ENCFF783KQI.mcool 2025-06-17 12:47:20,136::INFO::normalize_embeddings: Normalizing embeddings for ENCFF839VMS.mcool 2025-06-17 12:47:23,530::INFO::normalize_embeddings: Normalizing embeddings for ENCFF842DUO.mcool 2025-06-17 12:47:26,979::INFO::normalize_embeddings: Normalizing embeddings for ENCFF883XXW.mcool 2025-06-17 12:47:30,726::INFO::normalize_embeddings: Normalizing embeddings for ENCFF897KRI.mcool 2025-06-17 12:47:34,163::INFO::normalize_embeddings: Normalizing embeddings for ENCFF911AHQ.mcool 2025-06-17 12:47:37,382::INFO::normalize_embeddings: Normalizing embeddings for ENCFF925QIF.mcool 2025-06-17 12:47:40,902::INFO::normalize_embeddings: Normalizing embeddings for ENCFF943JRY.mcool 2025-06-17 12:47:44,283::INFO::normalize_embeddings: Normalizing embeddings for ENCFF952JZV.mcool 2025-06-17 12:47:47,702::INFO::normalize_embeddings: Normalizing embeddings for ENCFF958DWQ.mcool 2025-06-17 12:47:52,322::INFO::normalize_embeddings: Normalizing embeddings for ENCFF962EDB.mcool 2025-06-17 12:47:56,634::INFO::normalize_embeddings: Normalizing embeddings for ENCFF965BPU.mcool 2025-06-17 12:48:00,878::INFO::normalize_embeddings: Normalizing embeddings for ENCFF980NXK.mcool 2025-06-17 12:48:05,251::INFO::normalize_embeddings: Normalizing embeddings for ESC-FA-DSG-DdeI-DpnII-P1P2.hg38.mapq_30.1000.mcool 2025-06-17 12:48:09,607::INFO::normalize_embeddings: Normalizing embeddings for HB-FA-DSG-DdeI-DpnII-P1P2.hg38.mapq_30.1000.mcool 2025-06-17 12:48:13,800::INFO::normalize_embeddings: Normalizing embeddings for PGT_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:18,336::INFO::normalize_embeddings: Normalizing embeddings for PH_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:23,014::INFO::normalize_embeddings: Normalizing embeddings for PP_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:27,329::INFO::normalize_embeddings: Normalizing embeddings for SC_Beta_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:31,667::INFO::normalize_embeddings: Normalizing embeddings for day00_ESC_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:35,586::INFO::normalize_embeddings: Normalizing embeddings for day02_mesoderm_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:39,488::INFO::normalize_embeddings: Normalizing embeddings for day05_cardiacMesoderm_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:43,165::INFO::normalize_embeddings: Normalizing embeddings for day07_cardiacProgenitor_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:46,906::INFO::normalize_embeddings: Normalizing embeddings for day15_primativeCardiomyocyte_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:50,657::INFO::normalize_embeddings: Normalizing embeddings for day80_ventricularCardiomyocyte_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:54,536::INFO::normalize_embeddings: Normalizing embeddings for hESC_pool.hg38.mapq_30.100.mcool 2025-06-17 12:48:58,785::INFO::normalize_embeddings: Normalizing embeddings for iHEP-FA-DSG-DdeI-DpnII-P1P2.hg38.mapq_30.1000.mcool 2025-06-17 12:49:02,961::INFO::normalize_embeddings: Normalizing embeddings for mHEP-FA-DSG-DdeI-DpnII-P1P2.hg38.mapq_30.1000.mcool 2025-06-17 12:49:13,473::INFO::run: Components shape: (4493165, 32) 2025-06-17 12:49:13,474::INFO::run_kmeans: Running KMeans clustering with 5 clusters 2025-06-17 12:49:32,433::INFO::run_kmeans: Running KMeans clustering with 6 clusters 2025-06-17 12:49:57,732::INFO::run_kmeans: Running KMeans clustering with 8 clusters 2025-06-17 12:50:30,012::INFO::run_kmeans: Running KMeans clustering with 10 clusters 2025-06-17 12:51:06,720::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 14:32:29,019::INFO::run_leiden: Leiden clustering complete. 2025-06-17 14:32:32,321::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 16:08:59,266::INFO::run_leiden: Leiden clustering complete. 2025-06-17 16:09:02,270::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 17:34:30,414::INFO::run_leiden: Leiden clustering complete. 2025-06-17 17:34:33,412::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 18:58:57,692::INFO::run_leiden: Leiden clustering complete. 2025-06-17 18:59:00,720::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 20:26:14,758::INFO::run_leiden: Leiden clustering complete. 2025-06-17 20:26:17,350::INFO::run_leiden: Running Leiden clustering with 50 neighbors. 2025-06-17 21:49:25,072::INFO::run_leiden: Leiden clustering complete. 2025-06-17 21:49:28,037::INFO::run_umap: Running UMAP with 30 neighbors 2025-06-17 22:06:27,756::INFO::plot_scores: Plotting scores 2025-06-17 22:07:01,616::INFO::run: Saving embeddings to parquet and csv 2025-06-17 22:21:07,653::INFO::run: Finished running post-processing 2025-06-17 22:21:07,967::INFO::main: Finished jointly-hic