S3FS python, credential inline

raju picture raju · Aug 15, 2017 · Viewed 9.2k times · Source

I am trying to use python s3fs to read files in S3 AWS.

I could not find the code to put credential (Access key + Secret) into s3fs code.

Can anyone please help me how to set this info along with s3fs code.

import s3fs
fs = s3fs.S3FileSystem(anon=True)

I am currently on Windows 10.

Answer

omuthu picture omuthu · Aug 16, 2017

Try the below

import s3fs
fs = s3fs.S3FileSystem(anon=False, key='<Access Key>', secret='<Secret Key>')

Below are the list of parameters for S3FileSystem

Parameters
----------
anon : bool (False)
    Whether to use anonymous connection (public buckets only). If False,
    uses the key/secret given, or boto's credential resolver (environment
    variables, config files, EC2 IAM server, in that order)
key : string (None)
    If not anonymous, use this access key ID, if specified
secret : string (None)
    If not anonymous, use this secret access key, if specified
token : string (None)
    If not anonymous, use this security token, if specified
use_ssl : bool (True)
    Whether to use SSL in connections to S3; may be faster without, but
    insecure
s3_additional_kwargs : dict of parameters that are used when calling s3 api methods.
       Typically used for things like "ServerSideEncryption".
client_kwargs : dict of parameters for the boto3 client
requester_pays : bool (False)
    If RequesterPays buckets are supported.
default_block_size: None, int
    If given, the default block size value used for ``open()``, if no
    specific value is given at all time. The built-in default is 5MB.
default_fill_cache : Bool (True)
    Whether to use cache filling with open by default. Refer to
    ``S3File.open``.
config_kwargs : dict of parameters passed to ``botocore.client.Config``
kwargs : other parameters for boto3 session