Extract text from region in reading order

[dalanicolai]

Some very short background: I am trying to create a pymupdf backend for the amazing emacs pdf-tools package. Currently, pdf-tools uses a server written in C using the 'poppler' pdf-library. Creating a server using 'pymupdf' would make it much more hackable, and would make it much easier to add features like line annotations, pdf forms support etc. (I have already added a dirty hack for pymupdf support in the pymupdf-mode package, but that is no clean solution).
I have the pymupdf-server/backend already working b.t.w. (including annotations, but not yet for annotating text in reading order)

Now to get to the point: I would like to be able to select text regions in reading order (over multiple lines). Checking out the pdf-tools epdfinfo server, I have found that poppler 'simply' provides a function (poppler_page_get_selected_region) for this, but I have found no equivalent in pymupdf and neither in your pymupdf-utilities.

However, checking out the mupdf-gl reader (part of the mupdf library) annotation capabilities, I have found that that reader perfectly supports selecting text in reading order, so I would guess that the mupdf library somehow also already provides a function for this (which would probably be much faster than any hack written in python). Therefore, I was wondering if you are already aware of this, and if it would be possible to include this functionality in pymupdf (I hope you could determine the responsible function yourself, as I am no c-developer, but I would be happy to investigate it for you). It would be great if this would be possible (I figured you might be interested yourself also). Additionally, if pdf-tools would switch to the pymupdf library, this would be a great showcase for pymupdf as pdf-tools is really one of the most beautiful and most powerful, and wiht pymupdf for sure the most hackable, pdf readers available.

Here follow two screenshots of the functionality I would like to see, the first is of Emacs with pdf-tools using the current epdfinfo server, the second is of the 'mupdf' library its own mupdf-gl reader (in both images, the active areas are obtained using a single selection/mouse-drag). I only need to obtain the rectangles, coloring the active area I simply achieve (already) using the Pillow pdf library.

PDF-TOOLS

MUPDF-GL

Thank you!

[JorjMcKie]

There have been a number of text processing enhancements lately. For you purposes, let rect be the minimum rectangle containing start point A, end point B and which has the full page width, resp column text width. Then do words = page.get_text("words", sort=True, clip=rect).
From the start of the resulting list remove items as long as word[0] < A.x and remove words from the the end of the list as long as word[2] > B.x.
In your attached example, A corresponds to top-left of word "It", and B to bottom-right of word "editor.".
The resulting cleaned list then has the text " ".join([w[4] for w in words]).

[dalanicolai]

Thank you for your quick reply. Your solution works, but it is not ideal for the following two (or three) reasons:

1. It requires users to drag from before and above the upper left corner to after and below the lower right corner of the to be annotated text. In most other editors (like current pdf-tools and mupdf-gl), the user can just drag from left to right, from somewhere within the vertical location of the starting line to somewhere in the vertical location of the final line (this can be 'fixed' by writing some extra python code).
2. As far as I know, even with this trick to get the rectangles, it seems currently not possible to create a single annotation over multiple lines if the 'final markup' is not a single exact rectangle (the annotations in the examples of my initial message require multiple rectangles for the markup definition). Looking at the current code of pdf-tools, I have found that the annotations have a rectangle (named edges) definition and additionally a markup-edges definition, which can contain multiple rectangles for the exact markup. I guess it maps to the QuadPoints of the markup annotations as described on page 506 of the pdf specs (although it could be the AP streams as described on page 491 too). I could not find an option to set these using pymupdf. (I don't see how to fix this by using only the currently available pymupdf functions)
3. The selected area does not look very nice, although this is not very important, and also this is probably only because I am not used to the disconnected word-regions.

Here follows a screenshot of some active area using your trick (I did not yet bother to remove the 'items' from the left and the right, as I do not see how to use the result anyway, i.e. create the single annotation using multiple rectangles for markup)

[JorjMcKie]

Maybe we talked a little past each other:
If you were referring to adding highlight annotations between 2 given points (as opposed to extracting text from that region), things are easier text marking.
The call pattern is this add_highlight_annot(quads=None, start=None, stop=None, clip=None).
Just specify page.add_highlight_annot(start=A, stop=B) and you will get something like this:

Where the red dots are A resp. B.
For a multi-column page you would have to also specify clip to prevent the logic extending lines across columns.

[JorjMcKie]

So the effort reduces to find the A, B coordinates - and, potentially, the left and right borders of the page text columns we should restrict the highlight to.

BTW: the above works the same way for underline, strikethrough and squiggly annotations.

[JorjMcKie]

I am going to move this thread to the Discussions thread.