Simple ElasticSearch indexing integration for Django.
Project description
===========================
Django Simple Elasticsearch
===========================
.. image:: https://badge.fury.io/py/django-simple-elasticsearch.png
:target: http://badge.fury.io/py/django-simple-elasticsearch
.. image:: https://travis-ci.org/jaddison/django-simple-elasticsearch.png
:target: https://travis-ci.org/jaddison/django-simple-elasticsearch
.. image:: https://coveralls.io/repos/jaddison/django-simple-elasticsearch/badge.png
:target: https://coveralls.io/r/jaddison/django-simple-elasticsearch
This package provides a simple method of creating Elasticsearch indexes for
Django models.
-----
Versions
--------
Branch :code:`master` targets both Elasticsearch 2.x and 5.x and will receive new
features. Both `elasticsearch-py` 2.x and 5.x Python modules are currently
supported. `Documentation <http://django-simple-elasticsearch.readthedocs.io/>`_
Branch :code:`1.x` is the maintenance branch for the legacy 0.9.x versioned releases,
which targeted Elasticsearch versions less than 2.0. This branch is unlikely to
receive new features, but will receive required fixes.
`Documentation <http://django-simple-elasticsearch.readthedocs.io/en/1.x/>`_
**Using a version older than 0.9.0? Please be aware that as of v0.9.0, this package
has changed in a backwards-incompatible manner. Version 0.5 is deprecated and no
longer maintained.**
-----
Documentation
-------------
Visit the `django-simple-elasticsearch documentation on ReadTheDocs <http://django-simple-elasticsearch.readthedocs.org/>`_.
Features
--------
* class mixin with a set of :code:`@classmethods` used to handle:
* type mapping definition
* individual object indexing and deletion
* bulk object indexing
* model signal handlers for pre/post_save and pre/post_delete (optional)
* management command to handle index/type mapping initialization and bulk indexing
* uses Elasticsearch aliases to ease the burden of re-indexing
* small set of Django classes and functions to help deal with Elasticsearch querying
* base search form class to handle input validation, query preparation and response handling
* multi-search processor class to batch multiple Elasticsearch queries via :code:`_msearch`
* 'get' shortcut functions
* post index create/rebuild signals available to perform actions after certain stages (ie. add your own percolators)
Installation
------------
At the command line::
Configuring
-----------
Add the simple_elasticsearch application to your INSTALLED_APPS list::
INSTALLED_APPS = (
...
'simple_elasticsearch',
)
Add any models to `ELASTICSEARCH_TYPE_CLASSES` setting for indexing using **es_manage** management command::
ELASTICSEARCH_TYPE_CLASSES = [
'blog.models.BlogPost'
]
License
-------
**django-simple-elasticsearch** is licensed as free software under the BSD license.
Todo
----
* Review search classes - simplify functionality where possible. This may cause breaking changes.
* Tests. Write them.
* Documentation. Write it.
History
-------
2.2.1 (2017-11-15)
---------------------
* Fixing an issue with the recently released elasticsearch-py 6.0.0
2.2.0 (2017-07-17)
---------------------
* Addressing inability to index models with a non-integer PK field (ie. `UUIDField`) - added ability to order bulk queryset on an arbitrary model field.
2.1.7 (2017-03-21)
---------------------
* Allowing direct access (again) to underlying dict/list in `Result` and `Response` classes for serialization and other purposes.
2.1.5 (2017-03-20)
---------------------
* Response class is now MutableSequence based, giving it the properties of a `list`. Its `results` attribute is deprecated, as you can now iterate over the results with the response instance itself.
* Result class `results_meta` is deprecated. Use `meta` instead.
* `get_from_es_or_None` now returns a `Result` object instead of the raw Elasticsearch result, for consistency.
* `get_from_es_or_None` now catches only the Elasticsearch `NotFoundError` exception; previously it caught the more expansive `ElasticsearchException`, which could hide unrelated errors/issues.
2.1.4 (2017-03-12)
---------------------
* Result class is now MutableMapping based, giving it the properties of a `dict`. Its `data` attribute is deprecated.
2.1.3 (2017-03-11)
---------------------
* Minor changes to enable support for elasticsearch-py 5.x.
2.1.0 (2017-03-10)
---------------------
* Addressing a packaging problem which erroneously included pyc/__pycache__ files.
2.0.0 (2016-12-20)
---------------------
* **ALERT: this is a backwards incompatible release**; the old `1.x` (formerly `0.9.x`+) code is maintained on a separate branch for now.
* Added support for Django 1.10.
* Ported delete/cleanup functionality from `1.x`.
* Removed support for Django versions older than 1.8. The goal going forward will be to only support Django versions that the Django core team lists as supported.
* Removed elasticsearch-dsl support: responses and results are now represented by simpler internal representations; queries can ONLY be done via a `dict` form.
* Removed `ElasticsearchForm` - it is easy enough to build a form to validate search input and then form a query `dict` manually.
* Renamed `ElasticsearchIndexMixin` to `ElasticsearchTypeMixin`, seeing as the mixin represented an ES type mapping, not an actual index.
* Renamed `ElasticsearchProcessor` to `SimpleSearch`.
* Overall, this module has been greatly simplified.
1.0.0 (2016-12-20)
---------------------
* Updated 0.9.x codebase version to 1.0.0.
* Reversed decision on the deprecation of the 0.9.x codebase - it will be maintained in this new 1.x branch, although new functionality will mostly occur on newer releases.
* Adding cleanup command to remove unaliased indices.
* Added ELASTICSEARCH_DELETE_OLD_INDEXES setting to auto-remove after a rebuild.
* Thanks to Github user @jimjkelly for the index removal inspiration.
0.9.16 (2015-04-24)
---------------------
* Addressing Django 1.8 warnings.
0.9.15 (2015-01-31)
---------------------
* BUGFIX: Merging pull request from @key that addresses Python 3 support (management command now works).
0.9.14 (2015-01-31)
---------------------
* BUGFIX: Adding in missing `signals.py` file.
0.9.13 (2015-01-30)
---------------------
* Added in new `post_indices_create` and `post_indices_rebuild` signals to allow users to run actions at various points during the index creation and bulk indexing processes.
0.9.12 (2014-12-17)
---------------------
* fixed an issue where per-item request parameters were being added to the bulk data request JSON incorrectly. Tests updated.
0.9.11 (2014-12-08)
---------------------
* added warning if Django's DEBUG=True (causes out of memory errors on constrained
systems due to Django query caching)
* added index setting modification on rebuilding indices to remove replicas, lucene
segment merging and disabling the refresh interval - restoring the original
settings afterwards.
0.9.10 (2014-12-04)
---------------------
* added `page` and `page_size` validation in `add_search()`
0.9.9 (2014-11-24)
---------------------
* Renamed search form related classes - more breaking changes. Added in support
for Django's pagination classes (internal hack).
0.9.8 (2014-11-23)
---------------------
* Revamped search form related classes - includes breaking changes.
0.9.7 (2014-11-16)
---------------------
* Python3 supported mentioned in PyPi categorization; new testcases added. Minor
interface change (added `@classmethod`).
0.9.6 (2014-11-16)
---------------------
* Python 3.3+ support, modified (no new) testcases.
0.9.5 (2014-11-15)
---------------------
* Added in tox support, initial set of test cases and verified travis-ci is working.
0.9.2 (2014-11-12)
---------------------
* Fixed broken management command.
0.9.1 (2014-11-10)
---------------------
* Added missing management command module.
0.9.0 (2014-11-10)
---------------------
* In what will become version 1.0, this 0.9.x codebase is a revamp of the
original codebase (v0.5.x). Completely breaking over previous versions.
0.5.0 (2014-03-05)
---------------------
Final release in 0.x codebase - this old codebase is now unmaintained.
Django Simple Elasticsearch
===========================
.. image:: https://badge.fury.io/py/django-simple-elasticsearch.png
:target: http://badge.fury.io/py/django-simple-elasticsearch
.. image:: https://travis-ci.org/jaddison/django-simple-elasticsearch.png
:target: https://travis-ci.org/jaddison/django-simple-elasticsearch
.. image:: https://coveralls.io/repos/jaddison/django-simple-elasticsearch/badge.png
:target: https://coveralls.io/r/jaddison/django-simple-elasticsearch
This package provides a simple method of creating Elasticsearch indexes for
Django models.
-----
Versions
--------
Branch :code:`master` targets both Elasticsearch 2.x and 5.x and will receive new
features. Both `elasticsearch-py` 2.x and 5.x Python modules are currently
supported. `Documentation <http://django-simple-elasticsearch.readthedocs.io/>`_
Branch :code:`1.x` is the maintenance branch for the legacy 0.9.x versioned releases,
which targeted Elasticsearch versions less than 2.0. This branch is unlikely to
receive new features, but will receive required fixes.
`Documentation <http://django-simple-elasticsearch.readthedocs.io/en/1.x/>`_
**Using a version older than 0.9.0? Please be aware that as of v0.9.0, this package
has changed in a backwards-incompatible manner. Version 0.5 is deprecated and no
longer maintained.**
-----
Documentation
-------------
Visit the `django-simple-elasticsearch documentation on ReadTheDocs <http://django-simple-elasticsearch.readthedocs.org/>`_.
Features
--------
* class mixin with a set of :code:`@classmethods` used to handle:
* type mapping definition
* individual object indexing and deletion
* bulk object indexing
* model signal handlers for pre/post_save and pre/post_delete (optional)
* management command to handle index/type mapping initialization and bulk indexing
* uses Elasticsearch aliases to ease the burden of re-indexing
* small set of Django classes and functions to help deal with Elasticsearch querying
* base search form class to handle input validation, query preparation and response handling
* multi-search processor class to batch multiple Elasticsearch queries via :code:`_msearch`
* 'get' shortcut functions
* post index create/rebuild signals available to perform actions after certain stages (ie. add your own percolators)
Installation
------------
At the command line::
Configuring
-----------
Add the simple_elasticsearch application to your INSTALLED_APPS list::
INSTALLED_APPS = (
...
'simple_elasticsearch',
)
Add any models to `ELASTICSEARCH_TYPE_CLASSES` setting for indexing using **es_manage** management command::
ELASTICSEARCH_TYPE_CLASSES = [
'blog.models.BlogPost'
]
License
-------
**django-simple-elasticsearch** is licensed as free software under the BSD license.
Todo
----
* Review search classes - simplify functionality where possible. This may cause breaking changes.
* Tests. Write them.
* Documentation. Write it.
History
-------
2.2.1 (2017-11-15)
---------------------
* Fixing an issue with the recently released elasticsearch-py 6.0.0
2.2.0 (2017-07-17)
---------------------
* Addressing inability to index models with a non-integer PK field (ie. `UUIDField`) - added ability to order bulk queryset on an arbitrary model field.
2.1.7 (2017-03-21)
---------------------
* Allowing direct access (again) to underlying dict/list in `Result` and `Response` classes for serialization and other purposes.
2.1.5 (2017-03-20)
---------------------
* Response class is now MutableSequence based, giving it the properties of a `list`. Its `results` attribute is deprecated, as you can now iterate over the results with the response instance itself.
* Result class `results_meta` is deprecated. Use `meta` instead.
* `get_from_es_or_None` now returns a `Result` object instead of the raw Elasticsearch result, for consistency.
* `get_from_es_or_None` now catches only the Elasticsearch `NotFoundError` exception; previously it caught the more expansive `ElasticsearchException`, which could hide unrelated errors/issues.
2.1.4 (2017-03-12)
---------------------
* Result class is now MutableMapping based, giving it the properties of a `dict`. Its `data` attribute is deprecated.
2.1.3 (2017-03-11)
---------------------
* Minor changes to enable support for elasticsearch-py 5.x.
2.1.0 (2017-03-10)
---------------------
* Addressing a packaging problem which erroneously included pyc/__pycache__ files.
2.0.0 (2016-12-20)
---------------------
* **ALERT: this is a backwards incompatible release**; the old `1.x` (formerly `0.9.x`+) code is maintained on a separate branch for now.
* Added support for Django 1.10.
* Ported delete/cleanup functionality from `1.x`.
* Removed support for Django versions older than 1.8. The goal going forward will be to only support Django versions that the Django core team lists as supported.
* Removed elasticsearch-dsl support: responses and results are now represented by simpler internal representations; queries can ONLY be done via a `dict` form.
* Removed `ElasticsearchForm` - it is easy enough to build a form to validate search input and then form a query `dict` manually.
* Renamed `ElasticsearchIndexMixin` to `ElasticsearchTypeMixin`, seeing as the mixin represented an ES type mapping, not an actual index.
* Renamed `ElasticsearchProcessor` to `SimpleSearch`.
* Overall, this module has been greatly simplified.
1.0.0 (2016-12-20)
---------------------
* Updated 0.9.x codebase version to 1.0.0.
* Reversed decision on the deprecation of the 0.9.x codebase - it will be maintained in this new 1.x branch, although new functionality will mostly occur on newer releases.
* Adding cleanup command to remove unaliased indices.
* Added ELASTICSEARCH_DELETE_OLD_INDEXES setting to auto-remove after a rebuild.
* Thanks to Github user @jimjkelly for the index removal inspiration.
0.9.16 (2015-04-24)
---------------------
* Addressing Django 1.8 warnings.
0.9.15 (2015-01-31)
---------------------
* BUGFIX: Merging pull request from @key that addresses Python 3 support (management command now works).
0.9.14 (2015-01-31)
---------------------
* BUGFIX: Adding in missing `signals.py` file.
0.9.13 (2015-01-30)
---------------------
* Added in new `post_indices_create` and `post_indices_rebuild` signals to allow users to run actions at various points during the index creation and bulk indexing processes.
0.9.12 (2014-12-17)
---------------------
* fixed an issue where per-item request parameters were being added to the bulk data request JSON incorrectly. Tests updated.
0.9.11 (2014-12-08)
---------------------
* added warning if Django's DEBUG=True (causes out of memory errors on constrained
systems due to Django query caching)
* added index setting modification on rebuilding indices to remove replicas, lucene
segment merging and disabling the refresh interval - restoring the original
settings afterwards.
0.9.10 (2014-12-04)
---------------------
* added `page` and `page_size` validation in `add_search()`
0.9.9 (2014-11-24)
---------------------
* Renamed search form related classes - more breaking changes. Added in support
for Django's pagination classes (internal hack).
0.9.8 (2014-11-23)
---------------------
* Revamped search form related classes - includes breaking changes.
0.9.7 (2014-11-16)
---------------------
* Python3 supported mentioned in PyPi categorization; new testcases added. Minor
interface change (added `@classmethod`).
0.9.6 (2014-11-16)
---------------------
* Python 3.3+ support, modified (no new) testcases.
0.9.5 (2014-11-15)
---------------------
* Added in tox support, initial set of test cases and verified travis-ci is working.
0.9.2 (2014-11-12)
---------------------
* Fixed broken management command.
0.9.1 (2014-11-10)
---------------------
* Added missing management command module.
0.9.0 (2014-11-10)
---------------------
* In what will become version 1.0, this 0.9.x codebase is a revamp of the
original codebase (v0.5.x). Completely breaking over previous versions.
0.5.0 (2014-03-05)
---------------------
Final release in 0.x codebase - this old codebase is now unmaintained.
