Run#
- pyxel.run_mode(mode, detector, pipeline, override_dct=None, debug=False, with_inherited_coords=False)[source]#
Run a pipeline.
- Parameters:
mode (
Exposure,ObservationorCalibration) – Mode to execute.detector (
Detector) – This object is the container for all the data used for the models.pipeline (
DetectionPipeline) – This is the core algorithm of Pyxel. This pipeline contains all the models to run.override_dct (
dict, optional) – A dictionary of parameter(s) to override during processing.debug (
bool, default:False) – Add all intermediate steps into the results as aDataTree. This mode is used for debugging.with_inherited_coords (
bool, default:False) – Return the results a DataTree with better hierarchical format. This parameter is provisional.
Notes
Parameter
debugandwith_hiearchical_formatare not (yet) stable and may change in the future.- Returns:
- Raises:
TypeError – Raised if the
modeis not valid.NotImplementedError – Raised if parameter
debugis activated and mode is not anExposureobject.
Examples
Run an ‘Exposure’ pipeline
>>> import pyxel >>> config = pyxel.load("exposure_configuration.yaml") >>> data_tree = pyxel.run_mode( ... mode=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... with_inherited_coords=True, # with the new 'provisional' parameter ... override={ # optional ... "exposure.outputs.output_folder": "new_folder", ... "pipeline.photon_collection.load_image.arguments.image_file": "new_image.fits", ... }, ... ) >>> data_tree <xarray.DataTree> Group: / │ Dimensions: () │ Data variables: │ *empty* │ Attributes: │ pyxel version: 2.4.1+56.ga760893c.dirty │ running mode: Exposure ├── Group: /bucket │ │ Dimensions: (time: 54, y: 100, x: 100) │ │ Coordinates: │ │ * time (time) float64 0.02 0.06 0.12 0.2 0.3 ... 113.0 117.8 122.7 127.7 │ │ * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ │ * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ │ Data variables: │ │ photon (time, y, x) float64 4MB 85.0 120.0 109.0 ... 2.533e+04 2.51e+04 │ │ charge (time, y, x) float64 4MB 201.0 196.0 202.0 ... 2.543e+04 2.52e+04 │ │ pixel (time, y, x) float64 4MB 77.38 110.0 99.09 ... 2.406e+04 2.406e+04 │ │ signal (time, y, x) float64 4MB 0.0009377 0.001322 0.00133 ... 0.2968 0.2968 │ │ image (time, y, x) float64 4MB 16.0 22.0 22.0 ... 4.863e+03 4.863e+03 │ └── Group: /bucket/scene │ └── Group: /bucket/scene/list │ └── Group: /bucket/scene/list/0 │ Dimensions: (ref: 345, wavelength: 343) │ Coordinates: │ * ref (ref) int64 0 1 2 3 4 5 6 7 ... 337 338 339 340 341 342 343 344 │ * wavelength (wavelength) float64 336.0 338.0 340.0 ... 1.018e+03 1.02e+03 │ Data variables: │ x (ref) float64 3KB 2.057e+05 2.058e+05 ... 2.031e+05 2.03e+05 │ y (ref) float64 3KB 8.575e+04 8.58e+04 ... 8.795e+04 8.807e+04 │ weight (ref) float64 3KB 11.49 14.13 15.22 14.56 ... 15.21 11.51 8.727 │ flux (ref, wavelength) 1MB float64 0.03769 0.04137 ... 1.813 1.896 └── Group: /data ├── Group: /data/mean_variance │ └── Group: /data/mean_variance/image │ Dimensions: (mean: 54) │ Coordinates: │ * mean (mean) float64 19.64 38.7 57.77 ... 4.586e+03 4.682e+03 4.777e+03 │ Data variables: │ variance (mean) float64 432B 5.893 10.36 15.13 ... 1.235e+04 1.297e+04 1.342e+04 └── Group: /data/statistics └── Group: /data/statistics/pixel Dimensions: (time: 54) Coordinates: * time (time) float64 0.02 0.06 0.12 0.2 0.3 ... 113.0 117.8 122.7 127.7 Data variables: var (time) float64 432B 92.4 197.8 317.2 ... 3.027e+05 3.175e+05 3.286e+05 mean (time) float64 432B 94.64 189.1 283.5 ... 2.269e+04 2.316e+04 2.363e+04 min (time) float64 432B 63.39 134.9 220.3 ... 2.135e+04 2.193e+04 2.24e+04 max (time) float64 432B 134.8 248.1 359.7 ... 2.522e+04 2.569e+04 2.64e+04 count (time) float64 432B 1e+04 1e+04 1e+04 1e+04 ... 1e+04 1e+04 1e+04 1e+04
Run a ‘Calibration’ pipeline
>>> config = pyxel.load("calibration_configuration.yaml") >>> data = pyxel.run_mode( ... mode=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... ) >>> data DataTree('None', parent=None) │ Dimensions: (evolution: 5, island: 1, param_id: 4, individual: 10, │ processor: 10, readout_time: 1, y: 235, x: 1) │ Coordinates: │ * evolution (evolution) int64 0 1 2 3 4 │ * island (island) int64 0 │ * param_id (param_id) int64 0 1 2 3 │ * individual (individual) int64 0 1 2 3 4 5 6 7 8 9 │ * processor (processor) int64 0 1 2 3 4 5 6 7 8 9 │ * readout_time (readout_time) int64 1 │ * y (y) int64 2065 2066 2067 2068 ... 2296 2297 2298 2299 │ * x (x) int64 0 │ Data variables: │ champion_fitness (island, evolution) float64 3.271e+06 ... 4.641e+05 │ champion_decision (island, evolution, param_id) float64 -2.224 ... 3.662 │ champion_parameters (island, evolution, param_id) float64 0.00597 ... 4.... │ best_fitness (island, evolution, individual) float64 3.271e+06 ..... │ best_decision (island, evolution, individual, param_id) float64 -2... │ best_parameters (island, evolution, individual, param_id) float64 0.... │ simulated_photon (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 235, 1), meta=np.ndarray> │ simulated_charge (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 235, 1), meta=np.ndarray> │ simulated_pixel (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 235, 1), meta=np.ndarray> │ simulated_signal (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 235, 1), meta=np.ndarray> │ simulated_image (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 235, 1), meta=np.ndarray> │ target (processor, y, x) float64 13.75 0.4567 ... 0.2293 0.375 │ Attributes: │ num_islands: 1 │ population_size: 10 │ num_evolutions: 5 │ generations: 1 │ topology: fully_connected │ result_type: pixel └── DataTree('full_size') Dimensions: (island: 1, processor: 10, readout_time: 1, y: 2300, x: 1) Coordinates: * island (island) int64 0 * processor (processor) int64 0 1 2 3 4 5 6 7 8 9 * readout_time (readout_time) int64 1 * y (y) int64 0 1 2 3 4 5 6 ... 2294 2295 2296 2297 2298 2299 * x (x) int64 0 Data variables: simulated_photon (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 2300, 1), meta=np.ndarray> simulated_charge (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 2300, 1), meta=np.ndarray> simulated_pixel (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 2300, 1), meta=np.ndarray> simulated_signal (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 2300, 1), meta=np.ndarray> simulated_image (island, processor, readout_time, y, x) float64 dask.array<chunksize=(1, 1, 1, 2300, 1), meta=np.ndarray> target (processor, y, x) float64 0.0 0.4285 ... 0.2293 0.375
Run a pipeline with all intermediate steps
>>> results = pyxel.run_mode( ... mode=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... debug=True, ... ) >>> results["/intermediate"] DataTree('intermediate', parent="data") │ Dimensions: () │ Data variables: │ *empty* │ Attributes: │ long_name: Store all intermediate results modified along a pipeline └── DataTree('time_idx_0') │ Dimensions: () │ Data variables: │ *empty* │ Attributes: │ long_name: Pipeline for one unique time │ pipeline_count: 0 │ time: 1.0 s ├── DataTree('photon_collection') │ │ Dimensions: () │ │ Data variables: │ │ *empty* │ │ Attributes: │ │ long_name: Model group: 'photon_collection' │ └── DataTree('load_image') │ Dimensions: (y: 100, x: 100) │ Coordinates: │ * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ Data variables: │ photon (y, x) float64 1.515e+04 1.592e+04 ... 1.621e+04 1.621e+04 │ Attributes: │ long_name: Group: 'load_image' ├── DataTree('charge_generation') │ │ Dimensions: () │ │ Data variables: │ │ *empty* │ │ Attributes: │ │ long_name: Model group: 'charge_generation' │ └── DataTree('photoelectrons') │ Dimensions: (y: 100, x: 100) │ Coordinates: │ * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ Data variables: │ charge (y, x) float64 1.515e+04 1.592e+04 ... 1.621e+04 1.621e+04 │ Attributes: │ long_name: Group: 'photoelectrons' ├── DataTree('charge_collection') │ │ Dimensions: () │ │ Data variables: │ │ *empty* │ │ Attributes: │ │ long_name: Model group: 'charge_collection' │ └── DataTree('simple_collection') │ Dimensions: (y: 100, x: 100) │ Coordinates: │ * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ Data variables: │ pixel (y, x) float64 1.515e+04 1.592e+04 ... 1.621e+04 1.621e+04 │ Attributes: │ long_name: Group: 'simple_collection' ├── DataTree('charge_measurement') │ │ Dimensions: () │ │ Data variables: │ │ *empty* │ │ Attributes: │ │ long_name: Model group: 'charge_measurement' │ └── DataTree('simple_measurement') │ Dimensions: (y: 100, x: 100) │ Coordinates: │ * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 │ Data variables: │ signal (y, x) float64 0.04545 0.04776 0.04634 ... 0.05004 0.04862 0.04862 │ Attributes: │ long_name: Group: 'simple_measurement' └── DataTree('readout_electronics') │ Dimensions: () │ Data variables: │ *empty* │ Attributes: │ long_name: Model group: 'readout_electronics' └── DataTree('simple_adc') Dimensions: (y: 100, x: 100) Coordinates: * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 * x (x) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99 Data variables: image (y, x) uint32 298 314 304 304 304 314 ... 325 339 339 328 319 319 Attributes: long_name: Group: 'simple_adc'
- pyxel.exposure_mode(exposure, detector, pipeline)[source]#
Run an ‘exposure’ pipeline.
Deprecated since version 1.14: exposure_mode will be removed in pyxel 2.0.0, it is replaced by pyxel.run_mode.
For more information, see Exposure mode.
- Parameters:
exposure (
Exposure)detector (
Detector)pipeline (
DetectionPipeline)
- Returns:
Examples
Load a configuration file
>>> import pyxel >>> config = pyxel.load("configuration.yaml") >>> config Configuration(...)
Run an exposure pipeline
>>> dataset = pyxel.exposure_mode( ... exposure=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... ) >>> dataset <xarray.Dataset> Dimensions: (readout_time: 1, y: 450, x: 450) Coordinates: * readout_time (readout_time) int64 1 * y (y) int64 0 1 2 3 4 5 6 7 ... 442 443 444 445 446 447 448 449 * x (x) int64 0 1 2 3 4 5 6 7 ... 442 443 444 445 446 447 448 449 Data variables: image (readout_time, y, x) uint16 9475 9089 8912 ... 9226 9584 10079 signal (readout_time, y, x) float64 3.159 3.03 2.971 ... 3.195 3.36 pixel (readout_time, y, x) float64 1.053e+03 1.01e+03 ... 1.12e+03
- pyxel.observation_mode(observation, detector, pipeline)[source]#
Run an ‘observation’ pipeline.
Deprecated since version 1.14: observation_mode will be removed in pyxel 2.0.0, it is replaced by pyxel.run_mode.
For more information, see Observation mode.
- Parameters:
observation (
Observation)detector (
Detector)pipeline (
DetectionPipeline)
- Returns:
ObservationResult– Result.
Examples
Load a configuration file
>>> import pyxel >>> config = pyxel.load("configuration.yaml") >>> config Configuration(...)
Run an observation pipeline
>>> result = pyxel.observation_mode( ... exposure=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... ) >>> result ObservationResult(...)
- pyxel.calibration_mode(calibration, detector, pipeline, compute_and_save=True)[source]#
Run a ‘calibration’ pipeline.
Deprecated since version 1.14: calibration_mode will be removed in pyxel 2.0.0, it is replaced by pyxel.run_mode.
For more information, see Calibration mode.
- Parameters:
calibration (
Calibration)detector (
Detector)pipeline (
DetectionPipeline)compute_and_save (
bool)
- Returns:
Examples
Load a configuration file
>>> import pyxel >>> config = pyxel.load("configuration.yaml") >>> config Configuration(...)
Run a calibration pipeline
>>> ds, processors, logs, filenames = pyxel.calibration_mode( ... exposure=config.exposure, ... detector=config.detector, ... pipeline=config.pipeline, ... )
>>> ds <xarray.Dataset> Dimensions: (island: 2, evolution: 2, param_id: 4, id_processor: 1, readout_time: 1, y: 100, x: 100) Coordinates: * island (island) int64 1 2 * evolution (evolution) int64 1 2 * id_processor (id_processor) int64 0 * readout_time (readout_time) int64 1 * y (y) int64 0 1 2 3 4 5 6 7 8 ... 92 93 94 95 96 97 98 99 * x (x) int64 0 1 2 3 4 5 6 7 8 ... 92 93 94 95 96 97 98 99 * param_id (param_id) int64 0 1 2 3 Data variables: champion_fitness (island, evolution) float64 4.759e+06 ... 4.533e+06 champion_decision (island, evolution, param_id) float64 0.08016 ... 7.... champion_parameters (island, evolution, param_id) float64 0.08016 ... 7.... simulated_image (island, id_processor, readout_time, y, x) uint16 48... simulated_signal (island, id_processor, readout_time, y, x) float64 4... simulated_pixel (island, id_processor, readout_time, y, x) float64 6... target (id_processor, y, x) >f8 4.834e+03 ... 4.865e+03 Attributes: num_islands: 2 population_size: 20 num_evolutions: 2 generations: 5 topology: unconnected result_type: image
>>> processors island id_processor processor 0 0 0 Delayed('apply_parameters-c5da1649-766f-4ecb-a... 1 1 0 Delayed('apply_parameters-c16f998f-f52f-4beb-b...
>>> logs num_generations ... global_num_generations 0 1 ... 1 1 2 ... 2 2 3 ... 3 3 4 ... 4 4 5 ... 5 .. ... ... ... 15 1 ... 6 16 2 ... 7 17 3 ... 8 18 4 ... 9 19 5 ... 10
>>> filenames []
- pyxel.run(input_filename, override=None, random_seed=None)[source]#
Run a YAML configuration file.
For more information, see Running modes.
Examples
>>> import pyxel >>> pyxel.run("configuration.yaml")
- pyxel.show_versions()[source]#
Display the versions of Pyxel and its dependencies.
Examples
>>> import pyxel >>> pyxel.show_versions() INSTALLED VERSIONS ------------------ commit : cfb7ce2886d24f884699aafd1ad7dc6f8615252d version : 0.6+18.gcfb7ce2.dirty python : 3.8.5 | packaged by conda-forge | (default, Aug 29 2020, 01:18:42) [Clang 10.0.1 ] python-bits: 64 OS : Darwin OS-release : 19.5.0 machine : x86_64 processor : i386 byteorder : little LC_ALL : None LANG : None LOCALE : None.UTF-8 pyxel : 0.6+18.gcfb7ce2.dirty astropy : 4.0.1.post1 dask : 2.25.0 distributed: 2.25.0 h5py : 2.10.0 ipywidgets : 7.5.1 jupyter : installed jupyterlab : 2.2.7 matplotlib : 3.3.1 numba : 0.51.2 numpy : 1.19.1 pandas : 1.1.2 poppy : 0.9.1 pygmo : 2.15.0 scipy : 1.5.2 setuptools : 49.6.0.post20200814 pip : 20.2.3 conda : 4.8.4 black : 20.8b1 flake8 : 3.8.3 isort : 5.5.2 mypy : installed pytest : 6.0.1 sphinx : None