GPU optimizations

[RonShakutai]

Change Description

This PR introduce GPU optimizations for Gliner, Spacy, stanza and transformers.

Technical Implementation

• Added DeviceDetector singleton for automatic GPU detection and CUDA initialization
• Integrated GPU support into all NLP engines (SpacyNlpEngine, TransformersNlpEngine, StanzaNlpEngine, GLiNERRecognizer)
• Dependency optimization: Using cupy-cuda12x

This PR introduces GPU handling improvements for GLiNER, spaCy, Transformers, and Stanza NLP engines, optimizing GPU detection and utilization.

Reproduce Results

On this branch:

cd presidio-analyzer
poetry run python ../benchmark_presidio.py --engines spacy/transformers/gliner/stanza --sizes 50,500 --json gpu_results.json

Then download the script, switch to main branch, and re-run: pay attention to change the json results file name in the command.

Compare gpu_results.json vs main_results.json.

Results!

GLiNER - Big Improvement

Rows	Metric	Before Optimization	After Optimization	Improvement
50	Total Time	13.37s	2.73s	4.9x faster
50	Throughput	3.74 texts/sec	18.31 texts/sec	4.9x faster
50	Entities Found	241	245	+1.7%
500	Total Time	211.21s	31.34s	6.7x faster
500	Throughput	2.37 texts/sec	15.96 texts/sec	6.7x faster
500	Entities Found	2,435	2,469	+1.4%

---

Transformers - Big Improvement

Comparison of Transformers (StanfordAIMI/stanford-deidentifier-base)

Rows	Metric	Before Optimization	After Optimization	Improvement
50	Total Time	3.64s	0.77s	4.7x faster
50	Throughput	13.73 texts/sec	64.85 texts/sec	4.7x faster
50	Entities Found	273	273	0%
500	Total Time	76.78s	7.97s	9.6x faster
500	Throughput	6.51 texts/sec	62.73 texts/sec	9.6x faster
500	Entities Found	2,746	2,746	0%

---

Stanza - Big Improvement

Rows	Metric	Before Optimization	After Optimization	Change
50	Total Time	9.53s	7.57s	1.3x faster
50	Throughput	5.24 texts/sec	6.61 texts/sec	1.3x faster
50	Entities Found	253	253	0%
500	Total Time	141.98s	33.77s	4.2x faster
500	Throughput	3.52 texts/sec	14.81 texts/sec	4.2x faster
500	Entities Found	2,510	2,511	+0.04%

---

spaCy - No Change

Comparison of spaCy (en_core_web_lg) performance before and after GPU handling improvements.

Rows	Metric	Before Optimization	After Optimization	Change
50	Total Time	0.36s	0.54s	1.5x slower
50	Throughput	138.58 texts/sec	93.25 texts/sec	1.5x slower
50	Entities Found	235	235	0%
500	Total Time	2.97s	4.62s	1.6x slower
500	Throughput	168.60 texts/sec	108.17 texts/sec	1.6x slower
500	Entities Found	2,377	2,377	0%

Checklist

• I have reviewed the contribution guidelines
• I have signed the CLA (if required)
• My code includes unit tests
• All unit tests and lint checks pass locally
• My PR contains documentation updates / additions if required

[github-actions]

Coverage report (presidio-anonymizer)

This PR does not seem to contain any modification to coverable code.

[github-actions]

Coverage report (presidio-structured)

This PR does not seem to contain any modification to coverable code.

[github-actions]

Coverage report (presidio-cli)

This PR does not seem to contain any modification to coverable code.

[github-actions]

Coverage report (presidio-image-redactor)

This PR does not seem to contain any modification to coverable code.

[omri374]

Thanks! A long due proper support for GPU workloads. Left some comments to consider.

[omri374]

Consider putting the benchmark result files in a dedicated folder under docs, or omit them from the repo. There's a chance for this to become stale very quickly

[RonShakutai]

its only for the GPU tests. i think we will need something more organized later on.

[omri374]

Already called in the super

[RonShakutai]

fixed.

[omri374]

Would this install cuda? Will the work for CPU only machines?

[RonShakutai]

Yes, but I believe it should be in the new GPU section to give users more flexibility regarding GPU dependencies. and not install what not in use.

[RonShakutai]

Thanks! A long due proper support for GPU workloads. Left some comments to consider.
Hi @omri374

This PR is not ready for full review yet. I mainly want someone with a different GPU to run it and check whether the GPU-pref optimization improves performance on their setup.

Before finalizing, I’ll remove the benchmark script—I kept it only so others can measure GPU improvements on their machines.

I’m also considering adding an extra gpu section in the pyproject.toml, so people can either install a common set of GPU dependencies or use their own GPU libraries.

[github-actions]

Coverage report (presidio-analyzer)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-analyzer/presidio_analyzer/nlp_engine
__init__.py
device_detector.py
spacy_nlp_engine.py
stanza_nlp_engine.py					292
transformers_nlp_engine.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/ner
gliner_recognizer.py					116
Project Total

This report was generated by python-coverage-comment-action

[tamirkamara]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

[RonShakutai]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

Yes, the device detection is automatic and requires no code changes

[tamirkamara]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

Yes, the device detection is automatic and requires no code changes

Doesn't GPU require drivers on the OS level? Not sure we install those currently.
I also see you have instructions to do pip install which makes we think this won't work unless we do something for it. But this can be an separate issue/PR.

[RonShakutai]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

Yes, the device detection is automatic and requires no code changes

Doesn't GPU require drivers on the OS level? Not sure we install those currently. I also see you have instructions to do pip install which makes we think this won't work unless we do something for it. But this can be an separate issue/PR.

GPU execution still requires OS-level drivers (CUDA, NVIDIA runtime), which are outside the scope of this PR.

This PR focuses on two things:

Correct GPU usage in code paths for Stanza, spaCy, GLiNER, and Transformers, once a GPU is available.

Providing an optional gpu extra that installs commonly used CUDA-compatible Python dependencies, so users can “plug and go” in most setups.

CUDA versions and drivers are highly GPU-specific and must be installed by the GPU owner, who knows their hardware best. For that reason, we do not bundle or enforce CUDA drivers.

We only add a recommended GPU dependency set via an extra in pyproject.toml.
As a result, this change does not affect the Docker image and requires no Docker-level changes.

[omri374]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

Yes, the device detection is automatic and requires no code changes

Doesn't GPU require drivers on the OS level? Not sure we install those currently. I also see you have instructions to do pip install which makes we think this won't work unless we do something for it. But this can be an separate issue/PR.

GPU execution still requires OS-level drivers (CUDA, NVIDIA runtime), which are outside the scope of this PR.

This PR focuses on two things:

Correct GPU usage in code paths for Stanza, spaCy, GLiNER, and Transformers, once a GPU is available.

Providing an optional gpu extra that installs commonly used CUDA-compatible Python dependencies, so users can “plug and go” in most setups.

CUDA versions and drivers are highly GPU-specific and must be installed by the GPU owner, who knows their hardware best. For that reason, we do not bundle or enforce CUDA drivers.

We only add a recommended GPU dependency set via an extra in pyproject.toml. As a result, this change does not affect the Docker image and requires no Docker-level changes.

For this to work in Docker, we need two things: (1) code level adjusments, as done in this PR, (2) Dockerfile adjustments, as our current Dockerfile doesn't install cuda/cudnn packages (e.g. FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04).
I would suggest to finalize this PR to make sure that whoever wants to use GPU using the python package can, and later continue to providing GPU compatible Dockerfiles/images. WDYT?

[RonShakutai]

@RonShakutai Is this meant to be support inside the docker image without additional changes?

Yes, the device detection is automatic and requires no code changes

Doesn't GPU require drivers on the OS level? Not sure we install those currently. I also see you have instructions to do pip install which makes we think this won't work unless we do something for it. But this can be an separate issue/PR.

GPU execution still requires OS-level drivers (CUDA, NVIDIA runtime), which are outside the scope of this PR.
This PR focuses on two things:
Correct GPU usage in code paths for Stanza, spaCy, GLiNER, and Transformers, once a GPU is available.
Providing an optional gpu extra that installs commonly used CUDA-compatible Python dependencies, so users can “plug and go” in most setups.
CUDA versions and drivers are highly GPU-specific and must be installed by the GPU owner, who knows their hardware best. For that reason, we do not bundle or enforce CUDA drivers.
We only add a recommended GPU dependency set via an extra in pyproject.toml. As a result, this change does not affect the Docker image and requires no Docker-level changes.

For this to work in Docker, we need two things: (1) code level adjusments, as done in this PR, (2) Dockerfile adjustments, as our current Dockerfile doesn't install cuda/cudnn packages (e.g. FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04). I would suggest to finalize this PR to make sure that whoever wants to use GPU using the python package can, and later continue to providing GPU compatible Dockerfiles/images. WDYT?

Totally agree. @omri374
I think this PR should stay focused on the code-level GPU support (detection and correct applications of it in the different code components).
GPU drivers and CUDA versions are highly hardware-specific and should be installed by the environment owner.

For Docker concern that was raised correctly by @tamirkamara !, this is a separate concern.
We can address it later by providing GPU-enabled Docker images or a solution that allows choosing the appropriate base image at build/run time, depending on the available hardware.