No description
Find a file
Deathwish e0b0203eef
Merge pull request #4 from eligoenergy/BUGFIX-support_dotted_bucket_names
Add TLS domain validation logic needed to handle bucket names containing '.'.
2019-05-07 17:50:37 -05:00
bin initial version 2014-01-04 19:05:15 -06:00
docs initial version 2014-01-04 19:05:15 -06:00
s3fdw Add TLS domain validation logic needed to handle bucket names containing '.'. 2019-05-07 17:12:41 -05:00
tests initial version 2014-01-04 19:05:15 -06:00
.gitignore initial version 2014-01-04 19:05:15 -06:00
CHANGES.txt initial version 2014-01-04 19:05:15 -06:00
LICENSE.txt initial version 2014-01-04 19:05:15 -06:00
MANIFEST.in initial version 2014-01-04 19:05:15 -06:00
README.rst Various cleanups at README.rst 2017-10-06 10:48:28 -07:00
setup.py empty strings from CSVs are NULLs 2014-01-05 00:29:36 -06:00

============================================
S3 CSV Foreign Data Wrapper for PostgreSQL
============================================
This data wrapper adds the ability to perform 'select *' queries on
CSV files stored on the Amazon S3 file system.  This is meant to
replace s3_fdw_ which is not supported on PostgreSQL version 9.2+.

.. _s3_fdw: https://github.com/umitanuki/s3_fdw


Install multicorn
===========================================
First you need to install it (last command might need a sudo).

.. code:: bash
    git clone git@github.com:eligoenergy/s3csv_fdw.git
    cd s3csv_fdw
    python setup.py install
Then activate multicorn extension in your pg database

.. code:: sql
    CREATE EXTENSION multicorn;


Create Foreign Data Wrapper
============================================
Just paste this code to create server

.. code:: sql
    CREATE SERVER multicorn_csv FOREIGN DATA WRAPPER multicorn
    options (
    	wrapper 's3csvfdw.s3csvfdw.S3CsvFdw'
    );


Create Foreign Table
============================================
Replace the example fields with your info...

Example:

.. code:: sql
    CREATE FOREIGN TABLE test (
    	remote_filed1  character varying,
    	remote_field2  integer
    ) server multicorn_csv options(
    	bucket   'BUCKET',
    	filename 'FILENAME'
    );


Add user credentials
============================================
Store your aws credentials into a PostgreSQL user mapping.

Example:

.. code:: sql
    CREATE USER MAPPING FOR my_pg_user SERVER multicorn_dynamo OPTIONS (aws_access_key_id  'XXXXXXXXXXXXXXX',aws_secret_access_key  'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');


Perform queries
============================================
You have a PostgreSQL table now, for now, only read queries are working.

.. code:: sql
    SELECT * from test;


Credits
============================================
Christian Toivola (dev360_) wrote the code and submitted it as Multicorn
request here (https://github.com/Kozea/Multicorn/pull/49).  I just
packaged it up.

.. _dev360: https://github.com/dev360