Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.6-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd65cc3333ed684d84ffa54fa8e59d365869ecb1ea3dfb9232c7a75291455e6d |
|
MD5 | 24515f0d74424fe18caea2cbba2631ff |
|
BLAKE2b-256 | a6b5c4009d5405565d74db93ea8226a0ebb36eeab3ebd21078f950fe963a6a04 |
Hashes for asyncdb-2.7.6-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16a3c5b0b46643906eb2bc343828d1888b2512e33ae37feb4c65e5ca8df5e819 |
|
MD5 | 9f4fdec2af0652911f60bb05c5a2f7d7 |
|
BLAKE2b-256 | 1f0ba6c944be6c36087d574a8adaf77b9c09cbf858abc93d868241fc7e4942d5 |
Hashes for asyncdb-2.7.6-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ddfc1b52eb708fbb8b39423aaa14e0dfd396762cd3ba025332cc3b9f24c19d7 |
|
MD5 | f4c9e248dac40004eb73abf7072dfcea |
|
BLAKE2b-256 | 795e222874131ab6092335c99fa9c57508d37255c8b850000084e553bc934675 |
Hashes for asyncdb-2.7.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b771acfd439d5f125a708bc77a04cda5dcedf869cdb6af456c9893a4b41118a3 |
|
MD5 | f7acf8da3405b290149f4b84e058bbab |
|
BLAKE2b-256 | f9c6cfadd67d79bd5e6b79ec6c5d56fa8744499fc22fc0f1d3d163d2ba477360 |
Hashes for asyncdb-2.7.6-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ab8c8ebf0e95c9c33f44f03393ee7f14a9938586822653cbe04362d1b1f1e97 |
|
MD5 | 2d10289185cf722eeda4b828b4b59860 |
|
BLAKE2b-256 | c715693f15c139cd6dc5e10167eb1626bef51395d9726e7bdc4f848b78fdd53d |
Hashes for asyncdb-2.7.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 80142b20892e8f4961a3b7070c8e592a4bebd000d1736f0dfdabac66d31202f2 |
|
MD5 | 1d5175c7b7cd9523e0107f6b92a40441 |
|
BLAKE2b-256 | f322e78c718d3679d25bbbd503dd9d7c433c4c9c97112a2f00377bba94b554ff |
Hashes for asyncdb-2.7.6-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b589e174812e1efbdf94096d2318a4f9f5c99ad6085f62cfefd32b2573dbb1a |
|
MD5 | 5ca2ac9d60fc45e8c870f5bc97575ec3 |
|
BLAKE2b-256 | f055c088a8e42aedc2c11589bc843f1f0343b2521bddfdc4c1165db96f95d1e1 |
Hashes for asyncdb-2.7.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c0a27404b886b7d2ba59078b5d09fb6f4ced8f089e01e742b1d063b062708b3 |
|
MD5 | c1fc747a23bea6c438c402a73a559f91 |
|
BLAKE2b-256 | b4103282a765eeb1d8b10cc87a5a5a2489c84fe38fd10521a41d8bb57a955ef0 |
Hashes for asyncdb-2.7.6-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7972138647a8d07cdd059e3eef0c523463866ed256f341de0b8834983a6a23c |
|
MD5 | efc6bb56a3b3eae60d1bd14b5f52edef |
|
BLAKE2b-256 | bd9869d0ed412e299e994d6b29d3a37cf7ff76040464e7d3dc5b75231c79abca |
Hashes for asyncdb-2.7.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05181246bedaeed3ad6c3e0dd739c2fe6c518516677a015029b5406339209e82 |
|
MD5 | cca69815595140dad5dddcbfd14ca52e |
|
BLAKE2b-256 | a451973aba25447eddb86bc2934cc490b3f0711371170460a7190e4289d2ec1a |