Skip to content

IDEX L1a packet read failure #2538

@greglucas

Description

@greglucas

The L1a IDEX job failed on 12/2 with the following logs. This is generally an issue with the packet definitions, so we should look at what packets were transmitted on this date and whether we have everything we need to unpack the data properly.

Note from Ethan indicating this all looked nominal to the instrument team.

the data on 12/2 consisted of our event messages, and there have been no changes to the packet structure/definition over the last 1-2 years. 12/2 should be the same as our other daily event message transmissions.

INFO:imap_processing.cli:Interpreting dependency argument as a JSON file: imap_idex_l1a_all-20af83a4_20251202_v001.json
INFO:imap_data_access.io:Downloading file imap/dependency/idex/l1a/2025/12/imap_idex_l1a_all-20af83a4_20251202_v001.json from https://api.imap-mission.com/api-key/download/imap/dependency/idex/l1a/2025/12/imap_idex_l1a_all-20af83a4_20251202_v001.json to /app/data/imap/dependency/idex/l1a/2025/12/imap_idex_l1a_all-20af83a4_20251202_v001.json
INFO:imap_data_access.io:File /app/data/imap/dependency/idex/l1a/2025/12/imap_idex_l1a_all-20af83a4_20251202_v001.json downloaded successfully
INFO:imap_processing.cli:IMAP Processing Version: 1.0.3
INFO:imap_processing.cli:Processing Idex level l1a
INFO:imap_processing.cli:Beginning preprocessing (download dependencies)
INFO:imap_data_access.io:Downloading file imap/spice/lsk/naif0012.tls from https://api.imap-mission.com/api-key/download/imap/spice/lsk/naif0012.tls to /app/data/imap/spice/lsk/naif0012.tls
INFO:imap_data_access.io:File /app/data/imap/spice/lsk/naif0012.tls downloaded successfully
INFO:imap_data_access.io:Downloading file imap/spice/sclk/imap_sclk_0059.tsc from https://api.imap-mission.com/api-key/download/imap/spice/sclk/imap_sclk_0059.tsc to /app/data/imap/spice/sclk/imap_sclk_0059.tsc
INFO:imap_data_access.io:File /app/data/imap/spice/sclk/imap_sclk_0059.tsc downloaded successfully
INFO:imap_data_access.io:Downloading file imap/idex/l0/2025/12/imap_idex_l0_raw_20251202_v001.pkts from https://api.imap-mission.com/api-key/download/imap/idex/l0/2025/12/imap_idex_l0_raw_20251202_v001.pkts to /app/data/imap/idex/l0/2025/12/imap_idex_l0_raw_20251202_v001.pkts
INFO:imap_data_access.io:File /app/data/imap/idex/l0/2025/12/imap_idex_l0_raw_20251202_v001.pkts downloaded successfully
INFO:imap_processing.cli:Furnishing kernels: ['naif0012.tls', 'imap_sclk_0059.tsc']
INFO:imap_processing.cli:Beginning actual processing
INFO:space_packet_parser.definitions:Creating packet generator from a filelike object, <_io.BufferedReader name='/app/data/imap/idex/l0/2025/12/imap_idex_l0_raw_20251202_v001.pkts'>. Total length is 0 bytes
Traceback (most recent call last):
Processing IDEX l1a
  File "/usr/local/bin/imap_cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/imap_processing/cli.py", line 1511, in main
    instrument.process()
  File "/usr/local/lib/python3.12/site-packages/imap_processing/cli.py", line 444, in process
    products = self.do_processing(dependencies)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/imap_processing/cli.py", line 943, in do_processing
    datasets = PacketParser(science_files[0]).data
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/imap_processing/idex/idex_l1a.py", line 82, in __init__
    science_packets, raw_datset_by_apid, derived_datasets_by_apid = decom_packets(
                                                                    ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/imap_processing/idex/idex_l0.py", line 42, in decom_packets
    science_decom_packet_list = decom.decom_packets(packet_file, science_xtce_file)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/imap_processing/decom.py", line 36, in decom_packets
    return list(packet_generator)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/space_packet_parser/definitions.py", line 589, in packet_generator
    header = self._parse_header(header_bytes)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/space_packet_parser/definitions.py", line 374, in _parse_header
    header[item.name] = packets._extract_bits(packet_data, current_bit, item.nbits)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/space_packet_parser/packets.py", line 247, in _extract_bits
    return (value >> (len(data) * 8 - start_bit_within_byte - nbits)) & (2 ** nbits - 1)
            ~~~~~~^^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ValueError: negative shift count

Metadata

Metadata

Assignees

No one assigned

    Labels

    Ins: IDEXRelated to the IDEX instrumentLevel: L1Level 1 processingpacket parsingRelated to packet parsing or XTCE

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions