通过 python 子进程日志记录和回溯获取简洁的错误消息

问题描述 投票:0回答:1

我正在编写一个自动程序,我希望运行它来排除错误并收集信息。根据我当前的设置,我得到了一个循环实例的以下示例日志记录:

2024-03-21 10:14:44,843 - ERROR - error processing shot '20221201.033':
Traceback (most recent call last):
  File "/home/Desktop/Code/git/populate_database.py", line 76, in <module>
    raise Exception(f"n{stderr.decode()}")
Exception: n
    Fetching parlog information for shot 20221201.033 from archive:   0%|       | 0/5 [00:00<?, ?it/s]
    Fetching parlog information for shot 20221201.033 from archive:   0%|       | 0/5 [00:00<?, ?it/s]

Loading raw spectrometer data:   0%|        | 0/1016 [00:00<?, ?it/s]
Loading raw spectrometer data:  12%|█▏      | 117/1016 [00:00<00:00, 1164.76it/s]
Loading raw spectrometer data:  24%|██▎     | 239/1016 [00:00<00:00, 1194.36it/s]
Loading raw spectrometer data:  36%|███▌    | 361/1016 [00:00<00:00, 1203.52it/s]
Loading raw spectrometer data:  48%|████▊   | 485/1016 [00:00<00:00, 1217.46it/s]
Loading raw spectrometer data:  60%|█████▉  | 607/1016 [00:00<00:00, 1201.72it/s]
Loading raw spectrometer data:  72%|███████▏  | 729/1016 [00:00<00:00, 1206.46it/s]
Loading raw spectrometer data:  84%|████████▎ | 850/1016 [00:00<00:00, 1201.80it/s]
Loading raw spectrometer data:  96%|█████████▌| 976/1016 [00:00<00:00, 1218.04it/s]
Loading raw spectrometer data: 100%|██████████| 1016/1016 [00:00<00:00, 1209.91it/s]

    Status CXRS fits for ILS_Green:   0%|       | 0/10 [00:00<?, ?it/s]
    Status CXRS fits for ILS_Green:  10%|█      | 1/10 [00:45<06:46, 45.17s/it]
    Status CXRS fits for ILS_Green:  20%|██     | 2/10 [00:46<02:36, 19.55s/it]
    Status CXRS fits for ILS_Green:  30%|███    | 3/10 [00:48<01:20, 11.46s/it]
    Status CXRS fits for ILS_Green:  40%|████   | 4/10 [00:50<00:46,  7.77s/it]
    Status CXRS fits for ILS_Green:  50%|█████  | 5/10 [00:52<00:28,  5.62s/it]
    Status CXRS fits for ILS_Green:  60%|██████ | 6/10 [00:54<00:17,  4.29s/it]
    Status CXRS fits for ILS_Green:  70%|███████   | 7/10 [00:56<00:10,  3.50s/it]
    Status CXRS fits for ILS_Green:  80%|████████  | 8/10 [00:58<00:06,  3.03s/it]
    Status CXRS fits for ILS_Green:  90%|█████████ | 9/10 [00:59<00:02,  2.60s/it]
    Status CXRS fits for ILS_Green: 100%|██████████| 10/10 [01:01<00:00,  2.44s/it]
    Status CXRS fits for ILS_Green: 100%|██████████| 10/10 [01:01<00:00,  6.19s/it]

    Reading in neutral densities::   0%|        | 0/10 [00:00<?, ?it/s]
    Reading in neutral densities::  10%|█       | 1/10 [00:00<00:01,  5.89it/s]
    Reading in neutral densities:: 100%|██████████| 10/10 [00:00<00:00, 42.78it/s]
    Reading in neutral densities:: 100%|██████████| 10/10 [00:00<00:00, 35.99it/s]

    Mapping beam densities to cxrs timings    :   0%|       | 0/10 [00:00<?, ?it/s]
    Mapping beam densities to cxrs timings    : 100%|██████████| 10/10 [00:00<00:00, 2039.34it/s]

     Calculating reaction rates at profile time points:   0%|       | 0/10 [00:00<?, ?it/s]
     Calculating reaction rates at profile time points: 100%|██████████| 10/10 [00:00<00:00, 96.96it/s]
     Calculating reaction rates at profile time points: 100%|██████████| 10/10 [00:00<00:00, 96.79it/s]

     Writing rate information to each crossed cell:   0%|       | 0/10 [00:00<?, ?it/s]
     Writing rate information to each crossed cell:  30%|███    | 3/10 [00:00<00:00, 27.87it/s]
     Writing rate information to each crossed cell:  60%|██████ | 6/10 [00:00<00:00, 28.86it/s]
     Writing rate information to each crossed cell:  90%|█████████ | 9/10 [00:00<00:00, 29.17it/s]
     Writing rate information to each crossed cell: 100%|██████████| 10/10 [00:00<00:00, 29.02it/s]

     Mapping rates to CXRS timings    :   0%|       | 0/10 [00:00<?, ?it/s]
     Mapping rates to CXRS timings    : 100%|██████████| 10/10 [00:00<00:00, 2526.23it/s]

     Calculating impurity densities    :   0%|          | 0/10 [00:00<?, ?it/s]
     Calculating impurity densities    :  80%|████████  | 8/10 [00:00<00:00, 73.74it/s]
     Calculating impurity densities    : 100%|██████████| 10/10 [00:00<00:00, 73.56it/s]

     Updating Z_eff profile:   0%|          | 0/10 [00:00<?, ?it/s]
     Updating Z_eff profile:  80%|████████  | 8/10 [00:00<00:00, 73.27it/s]
     Updating Z_eff profile: 100%|██████████| 10/10 [00:00<00:00, 71.03it/s]

         Assessing convergence:   0%|       | 0/10 [00:00<?, ?it/s]
         Assessing convergence:   0%|       | 0/10 [00:00<?, ?it/s]

     Calculating reaction rates at profile time points:   0%|       | 0/10 [00:00<?, ?it/s]
     Calculating reaction rates at profile time points: 100%|██████████| 10/10 [00:00<00:00, 100.11it/s]

     Writing rate information to each crossed cell:   0%|       | 0/10 [00:00<?, ?it/s]
     Writing rate information to each crossed cell:  40%|████   | 4/10 [00:00<00:00, 30.03it/s]
     Writing rate information to each crossed cell:  80%|████████  | 8/10 [00:00<00:00, 30.17it/s]
     Writing rate information to each crossed cell: 100%|██████████| 10/10 [00:00<00:00, 30.07it/s]

     Mapping rates to CXRS timings    :   0%|       | 0/10 [00:00<?, ?it/s]
     Mapping rates to CXRS timings    : 100%|██████████| 10/10 [00:00<00:00, 3282.44it/s]

     Calculating impurity densities    :   0%|          | 0/10 [00:00<?, ?it/s]
     Calculating impurity densities    :  80%|████████  | 8/10 [00:00<00:00, 73.31it/s]
     Calculating impurity densities    : 100%|██████████| 10/10 [00:00<00:00, 73.30it/s]

     Updating Z_eff profile:   0%|          | 0/10 [00:00<?, ?it/s]
     Updating Z_eff profile:  80%|████████  | 8/10 [00:00<00:00, 71.31it/s]
     Updating Z_eff profile: 100%|██████████| 10/10 [00:00<00:00, 71.29it/s]

         Assessing convergence:   0%|       | 0/10 [00:00<?, ?it/s]
         Assessing convergence:  50%|█████  | 5/10 [00:00<00:00, 58743.75it/s]

     Calculating reaction rates at profile time points:   0%|       | 0/10 [00:00<?, ?it/s]
     Calculating reaction rates at profile time points: 100%|██████████| 10/10 [00:00<00:00, 100.53it/s]

     Writing rate information to each crossed cell:   0%|       | 0/10 [00:00<?, ?it/s]
     Writing rate information to each crossed cell:  40%|████   | 4/10 [00:00<00:00, 30.29it/s]
     Writing rate information to each crossed cell:  80%|████████  | 8/10 [00:00<00:00, 30.09it/s]
     Writing rate information to each crossed cell: 100%|██████████| 10/10 [00:00<00:00, 30.17it/s]

     Mapping rates to CXRS timings    :   0%|       | 0/10 [00:00<?, ?it/s]
     Mapping rates to CXRS timings    : 100%|██████████| 10/10 [00:00<00:00, 3360.28it/s]

     Calculating impurity densities    :   0%|          | 0/10 [00:00<?, ?it/s]
     Calculating impurity densities    :  80%|████████  | 8/10 [00:00<00:00, 73.92it/s]
     Calculating impurity densities    : 100%|██████████| 10/10 [00:00<00:00, 73.91it/s]

     Updating Z_eff profile:   0%|          | 0/10 [00:00<?, ?it/s]
     Updating Z_eff profile:  20%|██        | 2/10 [00:00<00:00,  8.86it/s]
     Updating Z_eff profile: 100%|██████████| 10/10 [00:00<00:00, 34.81it/s]
     Updating Z_eff profile: 100%|██████████| 10/10 [00:00<00:00, 29.59it/s]

         Assessing convergence:   0%|       | 0/10 [00:00<?, ?it/s]
         Assessing convergence: 100%|██████████| 10/10 [00:00<00:00, 84054.19it/s]
Traceback (most recent call last):
  File "/home/Desktop/Code/git/cxrs_fidasim/Examples/Program_runfiles/20221201.033.py", line 37, in <module>
    data.run(
  File "/home/Desktop/Code/git/cxrs_fidasim/cxrs_eval.py", line 348, in run
    self.run_post_methods()
  File "/home/Desktop/Code/git/cxrs_fidasim/cxrs_eval.py", line 469, in run_post_methods
    _store_grad_length(
  File "/home/Desktop/Code/git/cxrs_fidasim/utilities/write_locally.py", line 103, in _store_grad_length
    dictionary_added["tau_e"] = _get_energy_confinement_time(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Desktop/Code/git/cxrs_fidasim/utilities/fetch_data.py", line 293, in _get_energy_confinement_time
    w_dia_spline = UnivariateSpline(t_w_dia, w_dia, k=4, s=0)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anaconda3/lib/python3.11/site-packages/scipy/interpolate/_fitpack2.py", line 236, in __init__
    data = dfitpack.fpcurf0(x, y, k, w=w, xb=bbox[0],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dfitpack.error: (m>k) failed for hidden m: fpcurf0:m=0

这是完整的控制台,我只对记录包含实际错误的以下部分感兴趣:

Traceback (most recent call last):
  File "/home/Desktop/Code/git/cxrs_fidasim/Examples/Program_runfiles/20221201.033.py", line 37, in <module>
    data.run(
  File "/home/Desktop/Code/git/cxrs_fidasim/cxrs_eval.py", line 348, in run
    self.run_post_methods()
  File "/home/Desktop/Code/git/cxrs_fidasim/cxrs_eval.py", line 469, in run_post_methods
    _store_grad_length(
  File "/home/Desktop/Code/git/cxrs_fidasim/utilities/write_locally.py", line 103, in _store_grad_length
    dictionary_added["tau_e"] = _get_energy_confinement_time(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Desktop/Code/git/cxrs_fidasim/utilities/fetch_data.py", line 293, in _get_energy_confinement_time
    w_dia_spline = UnivariateSpline(t_w_dia, w_dia, k=4, s=0)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anaconda3/lib/python3.11/site-packages/scipy/interpolate/_fitpack2.py", line 236, in __init__
    data = dfitpack.fpcurf0(x, y, k, w=w, xb=bbox[0],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dfitpack.error: (m>k) failed for hidden m: fpcurf0:m=0

我当前的代码由以下

try
except
方法组成:

from subprocess import Popen, PIPE, STDOUT, DEVNULL

logging.basicConfig(filename='error.log', level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s')

   try:
        # run the file for the shot with error handling
        process = Popen(["python", dest_file], stdout=DEVNULL, stderr=PIPE)

        # wait while file is running
        process.wait()
     
        # get the console error of the subprocess
        stderr = process.communicate()[1]

        # close all figures and clear console
        plt.close()
        os.system('clear')
     
        # if we encountered an error raise it
        if stderr:
            raise Exception(f"n{stderr.decode()}")
    
    # except the raised error and log the error
    except Exception as e:
        # Log the shot name along with the traceback info
        logging.error(f"error processing shot '{shot_name}':", exc_info=True)

如何过滤掉相关部分并记录下来?

python logging subprocess traceback
1个回答
0
投票

鉴于您看起来只对任何 Python 回溯感兴趣,并且可能这将是最后打印的内容,您可以使用

rpartition
Traceback (most recent call last):
标头的最后一个实例处拆分文本:

if stderr:
    error = stderr.decode()
    error = error.rpartition("Traceback (most recent call last):\n")[-1]
    raise Exception(error)
© www.soinside.com 2019 - 2024. All rights reserved.