Add AQ support in thin mode for single enqueue and dequeue of RAW and

Oracle object payload types (#437).

Author: anthony-tuininga

doc/src/release_notes.rst

@@ -17,6 +17,8 @@ oracledb 3.0.0 (TBD)
 Thin Mode Changes
 +++++++++++++++++
 
+#)  Added :ref:`Oracle Advanced Queuing <aqusermanual>` support for single
+    enqueue and dequeue of RAW and Oracle object payload types.
 #)  Added namespace package :ref:`oracledb.plugins <plugins>` for plugins that
     can be used to extend the capability of python-oracledb.
 #)  Added support for property :attr:`ConnectionPool.max_lifetime_session`

doc/src/user_guide/appendix_a.rst

@@ -248,7 +248,7 @@ see :ref:`driverdiff` and :ref:`compatibility`.
       - Yes
       - Yes
     * - Oracle Transactional Event Queues and Advanced Queuing (AQ) (see :ref:`aqusermanual`)
-      - No
+      - Yes - RAW and named Oracle object payloads
       - Yes
       - Yes
     * - Call timeouts (see :attr:`Connection.call_timeout`)

doc/src/user_guide/aq.rst

@@ -15,10 +15,6 @@ receiving of various payloads, such as RAW values, JSON, JMS, and objects.
 Transactional Event Queues use a highly optimized implementation of Advanced
 Queuing. They were previously called AQ Sharded Queues.
 
-.. note::
-
-    TxEventQ and AQ Classic queues are only supported in python-oracledb Thick
-    mode.  See :ref:`enablingthick`.
 
 Python-oracledb API calls are the same for Transactional Event Queues and
 Classic Queues, however there are differences in support for some payload
@@ -31,11 +27,18 @@ types.
 - The JSON payload requires Oracle Client libraries 21c (or later) and Oracle
   Database 21c (or later).
 
-There are examples of AQ Classic Queues in the `GitHub examples
+JSON and JMS payloads, array message queuing and dequeuing operations, and
+:ref:`Recipient Lists <reciplists>` are only supported in python-oracledb
+:ref:`Thick mode <enablingthick>`.
+
+There are examples of AQ Classic Queues in the `GitHub samples
 <https://github.com/oracle/python-oracledb/tree/main/samples>`__ directory.
 
 **Transactional Event Queue Support**
 
+Transactional Event Queues are only supported in python-oracledb :ref:`Thick
+mode <enablingthick>`.
+
 - RAW and named Oracle object payloads are supported for single and array
   message enqueuing and dequeuing when using Oracle Client 19c (or later) and
   connected to Oracle Database 19c (or later).
@@ -55,7 +58,15 @@ Creating a Queue
 
 Before being used in applications, queues need to be created in the database.
 
-**Using RAW Payloads**
+To experiment with queueing, you can grant yourself privileges, for example in
+SQL*Plus as a DBA user:
+
+.. code-block:: sql
+
+    grant aq_administrator_role, aq_user_role to &&username;
+    grant execute on dbms_aq to &&username;
+
+**Creating RAW Payload Queues**
 
 To use SQL*Plus to create a Classic Queue for the RAW payload which is suitable
 for sending string or bytes messages:
@@ -79,7 +90,7 @@ To create a Transactional Event Queue for RAW payloads:
     end;
     /
 
-**Using JSON Payloads**
+**Creating JSON Payload Queues**
 
 Queues can also be created for JSON payloads. For example, to create a Classic
 Queue in SQL*Plus:
@@ -99,7 +110,7 @@ Enqueuing Messages
 To send messages in Python, you connect and get a :ref:`queue <queue>`. The
 queue can then be used for enqueuing, dequeuing, or for both.
 
-**Using RAW Payloads**
+**Enqueuing RAW Payloads**
 
 You can connect to the database and get the queue that was created with RAW
 payload type by using:
@@ -123,13 +134,14 @@ messages:
     connection.commit()
 
 Since the queue is a RAW queue, strings are internally encoded to bytes using
-``message.encode()`` before being enqueued.
+`encode() <https://docs.python.org/3/library/stdtypes.html#str.encode>`__
+before being enqueued.
 
-The use of :meth:`~Connection.commit()` means that messages are sent only when
-any database transaction related to them is committed. This behavior can be
-altered, see :ref:`aqoptions`.
+The use of :meth:`~Connection.commit()` allows messages to be sent only when
+any database transaction related to them is committed. This default behavior
+can be altered, see :ref:`aqoptions`.
 
-**Using JSON Payloads**
+**Enqueuing JSON Payloads**
 
 You can connect to the database and get the queue that was created with JSON
 payload type by using:
@@ -162,9 +174,11 @@ Dequeuing Messages
 ==================
 
 Dequeuing is performed similarly. To dequeue a message call the method
-:meth:`~Queue.deqone()` as shown in the examples below.
+:meth:`~Queue.deqone()` as shown in the examples below.  This returns a
+:ref:`MessageProperties <msgproperties>` object containing the message payload
+and related attributes.
 
-**Using RAW Payloads**
+**Dequeuing RAW Payloads**
 
 .. code-block:: python
 
@@ -174,9 +188,21 @@ Dequeuing is performed similarly. To dequeue a message call the method
     print(message.payload.decode())
 
 Note that if the message is expected to be a string, the bytes must be decoded
-by the application using ``message.payload.decode()``, as shown.
+by the application using `decode()
+<https://docs.python.org/3/library/stdtypes.html#bytes.decode>`__, as shown.
+
+If there are no messages in the queue, :meth:`~Queue.deqone()` will wait for
+one to be enqueued.  This default behavior can be altered, see
+:ref:`aqoptions`.
+
+Various :ref:`message properties <msgproperties>` can be accessed.  For example
+to show the :attr:`~MessageProperties.msgid` of a dequeued message:
+
+.. code-block:: python
+
+    print(message.msgid.hex())
 
-**Using JSON Payloads**
+**Dequeuing JSON Payloads**
 
 .. code-block:: python
 

samples/multi_consumer_aq.py

@@ -1,5 +1,5 @@
 # -----------------------------------------------------------------------------
-# Copyright (c) 2020, 2023, Oracle and/or its affiliates.
+# Copyright (c) 2020, 2025, Oracle and/or its affiliates.
 #
 # Portions Copyright 2007-2015, Anthony Tuininga. All rights reserved.
 #
@@ -37,8 +37,9 @@
 import oracledb
 import sample_env
 
-# this script is currently only supported in python-oracledb thick mode
-oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
+# determine whether to use python-oracledb thin mode or thick mode
+if not sample_env.get_is_thin():
+    oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
 
 QUEUE_NAME = "DEMO_RAW_QUEUE_MULTI"
 PAYLOAD_DATA = [

samples/object_aq.py

@@ -1,5 +1,5 @@
 # -----------------------------------------------------------------------------
-# Copyright (c) 2016, 2023, Oracle and/or its affiliates.
+# Copyright (c) 2016, 2025, Oracle and/or its affiliates.
 #
 # Portions Copyright 2007-2015, Anthony Tuininga. All rights reserved.
 #
@@ -39,8 +39,9 @@
 import oracledb
 import sample_env
 
-# this script is currently only supported in python-oracledb thick mode
-oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
+# determine whether to use python-oracledb thin mode or thick mode
+if not sample_env.get_is_thin():
+    oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
 
 BOOK_TYPE_NAME = "UDT_BOOK"
 QUEUE_NAME = "DEMO_BOOK_QUEUE"

samples/raw_aq.py

@@ -1,5 +1,5 @@
 # -----------------------------------------------------------------------------
-# Copyright (c) 2019, 2023, Oracle and/or its affiliates.
+# Copyright (c) 2019, 2025, Oracle and/or its affiliates.
 #
 # Portions Copyright 2007-2015, Anthony Tuininga. All rights reserved.
 #
@@ -37,8 +37,9 @@
 import oracledb
 import sample_env
 
-# this script is currently only supported in python-oracledb thick mode
-oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
+# determine whether to use python-oracledb thin mode or thick mode
+if not sample_env.get_is_thin():
+    oracledb.init_oracle_client(lib_dir=sample_env.get_oracle_client())
 
 QUEUE_NAME = "DEMO_RAW_QUEUE"
 PAYLOAD_DATA = [

src/oracledb/base_impl.pxd

@@ -317,6 +317,7 @@ cdef class Buffer:
                                bint write_length=*) except -1
     cdef int write_oracle_number(self, bytes num_bytes) except -1
     cdef int write_raw(self, const char_type *data, ssize_t length) except -1
+    cdef int write_sb4(self, int32_t value) except -1
     cdef int write_str(self, str value) except -1
     cdef int write_uint8(self, uint8_t value) except -1
     cdef int write_uint16be(self, uint16_t value) except -1
@@ -955,10 +956,13 @@ cdef object convert_oracle_data_to_python(OracleMetadata from_metadata,
                                           OracleData* data,
                                           const char* encoding_errors,
                                           bint from_dbobject)
+cdef object convert_date_to_python(OracleDataBuffer *buffer)
 cdef uint16_t decode_uint16be(const char_type *buf)
 cdef uint32_t decode_uint32be(const char_type *buf)
 cdef uint16_t decode_uint16le(const char_type *buf)
 cdef uint64_t decode_uint64be(const char_type *buf)
+cdef int decode_date(const uint8_t* ptr, ssize_t num_bytes,
+                     OracleDataBuffer *buffer)
 cdef void encode_uint16be(char_type *buf, uint16_t value)
 cdef void encode_uint16le(char_type *buf, uint16_t value)
 cdef void encode_uint32be(char_type *buf, uint32_t value)

src/oracledb/errors.py

@@ -358,6 +358,7 @@ def _raise_not_supported(feature: str) -> None:
 ERR_UNKNOWN_SERVER_PIGGYBACK = 5009
 ERR_UNKNOWN_TRANSACTION_STATE = 5010
 ERR_UNEXPECTED_PIPELINE_FAILURE = 5011
+ERR_NOT_IMPLEMENTED = 5012
 
 # error numbers that result in OperationalError
 ERR_LISTENER_REFUSED_CONNECTION = 6000
@@ -713,6 +714,7 @@ def _raise_not_supported(feature: str) -> None:
     ERR_NO_STATEMENT_PREPARED: "statement must be prepared first",
     ERR_NOT_A_QUERY: "the executed statement does not return rows",
     ERR_NOT_CONNECTED: "not connected to database",
+    ERR_NOT_IMPLEMENTED: "not implemented",
     ERR_NUMBER_STRING_OF_ZERO_LENGTH: "invalid number: zero length string",
     ERR_NUMBER_STRING_TOO_LONG: "invalid number: string too long",
     ERR_NUMBER_WITH_EMPTY_EXPONENT: "invalid number: empty exponent",

src/oracledb/impl/base/buffer.pyx

@@ -1,5 +1,5 @@
 #------------------------------------------------------------------------------
-# Copyright (c) 2020, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2020, 2025, Oracle and/or its affiliates.
 #
 # This software is dual-licensed to you under the Universal Permissive License
 # (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl and Apache License
@@ -791,6 +791,26 @@ cdef class Buffer:
             length -= bytes_to_write
             data += bytes_to_write
 
+    cdef int write_sb4(self, int32_t value) except -1:
+        """
+        Writes a 32-bit signed integer to the buffer in universal format.
+        """
+        cdef uint8_t sign = 0
+        if value < 0:
+            value = -value
+            sign = 0x80
+        if value == 0:
+            self.write_uint8(0)
+        elif value <= UINT8_MAX:
+            self.write_uint8(1 | sign)
+            self.write_uint8(<uint8_t> value)
+        elif value <= UINT16_MAX:
+            self.write_uint8(2 | sign)
+            self.write_uint16be(<uint16_t> value)
+        else:
+            self.write_uint8(4 | sign)
+            self.write_uint32be(value)
+
     cdef int write_str(self, str value) except -1:
         """
         Writes a string to the buffer as UTF-8 encoded bytes.

src/oracledb/impl/base/connection.pyx

@@ -1,5 +1,5 @@
 #------------------------------------------------------------------------------
-# Copyright (c) 2020, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2020, 2025, Oracle and/or its affiliates.
 #
 # This software is dual-licensed to you under the Universal Permissive License
 # (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl and Apache License
@@ -225,6 +225,9 @@ cdef class BaseConnImpl:
         cursor_impl.prefetchrows = C_DEFAULTS.prefetchrows
         return cursor_impl
 
+    def create_msg_props_impl(self):
+        errors._raise_not_supported("creating a message property object")
+
     def create_queue_impl(self):
         errors._raise_not_supported("creating a queue")