Skip to content

DefinedNamespaceMeta.__getitem__ is slow #1717

@anatoly-scherbakov

Description

@anatoly-scherbakov

An attempt to process https://schema.org vocabulary via OWL RL library takes approximately 1 minute 50 seconds on my machine: https://gist.github.com/anatoly-scherbakov/6deac9b7540ecacc723603d83d2d9cee

According to the profiler output, the functions that take a lot of time is __getitem__ method of DefinedNamespaceMeta class:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  8510246   14.562    0.000   26.914    0.000 __init__.py:216(__contains__)
  8510246   11.345    0.000   62.434    0.000 __init__.py:190(__getitem__)

I believe __getitem__ calls __contains__. I propose to cache __getitem__() to trade some RAM for better performance. After wrapping this into @lru_cache my result is this:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  9383995    5.253    0.000    5.253    0.000 term.py:144(__eq__)
  4223629    5.252    0.000   15.734    0.000 memory.py:305(triples)
  4223629    3.033    0.000   19.109    0.000 graph.py:453(triples)
  8510244    2.662    0.000    2.662    0.000 __init__.py:206(__getattr__)
  2459789    1.992    0.000    3.496    0.000 memory.py:496(__get_context_for_triple)
   187389    1.950    0.000    5.611    0.000 OWLRL.py:854(_schema_vocabulary)
  1797391    1.821    0.000    3.130    0.000 memory.py:521(__ctx_to_str)
   187389    1.632    0.000   18.468    0.000 OWLRL.py:369(_equality)
   187389    1.513    0.000    3.594    0.000 OWLRL.py:435(_properties)
   187389    1.490    0.000    3.139    0.000 OWLRL.py:637(_classes)

and total execution time is down to approximately 40 seconds. We can see now that the concerned methods aren't even in the top 10.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions