What is AWS Transfer family?

The AWS Transfer Family provides fully managed support for file transfers directly into and out of Amazon S3 or Amazon EFS. With support for Secure File Transfer Protocol (SFTP), File Transfer Protocol over SSL (FTPS), and File Transfer Protocol (FTP), the AWS Transfer Family helps you seamlessly migrate your file transfer workflows to AWS by integrating with existing authentication systems, and providing DNS routing with Amazon Route 53 so nothing changes for your customers and partners, or their applications. With your data in Amazon S3 or Amazon EFS, you can use it with AWS services for processing, analytics, machine learning, archiving, as well as home directories and developer tools. Getting started with the AWS Transfer Family is easy; there is no infrastructure to buy and set up.

• It can be used to transfer files into and out of S3 and EFS
• SFTP, FTPS, and FTP protocols are supported

How do we use it?

Demo

Visit https://ap-northeast-1.console.aws.amazon.com/transfer/home?region=ap-northeast-1#/

→ Click Create server

Set up configurations

Choose protocols

We can select one of the following options

• SFTP (SSH File Transfer Protocol)
• FTPS (File Transfer Protocol Secure)
• FTP (File Transfer Protocol)

Check SFTP (SSH File Transfer Protocol) - file transfer over Secure Shell and click Next

Choose an identity provider

We can select one of the following protocols

• Service managed
• Custom

Check Service managed and click Next

Choose an endpoint

We need to configure endpoint type and custom hostname. We have the following options for an endpoint.

• Publicly accessible
• VPC hosted

Check Publicly accessible and click Next

Choose a domain

We can use S3 and EFS as the destination or starting point of transfer of files. Click Amazon S3 and click Next.

Configure additional details

We can attach an existing role or create a new role to allow the transfer family to put logs in CloudWatch Logs. Choose to Create a new role and click Next

Review and create

Click Create server below.

Check on console

Add a service managed user

Click Server ID of the server that you want to add a user to. Click Add user.

User configuration

Attach to the role AWSTransferLoggingAccess

Input Username transfer-family-test, choose the role AWSTransferLoggingAccess. Then, choose S3 bucket where the files are transferred and

Generate SSH key

> ssh-keygen -P "" -m PEM -f ~/.ssh/transfer-family-test
Generating public/private rsa key pair.
Your identification has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.
Your public key has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.pub.
The key fingerprint is:
SHA256:zsBLQFnZn7PL+Q8ioX68yge7Nqo2lpCAgE4VcvDzM5A [email protected]
The key's randomart image is:
+---[RSA 3072]----+
|.oo+oo.o         |
|o.+.o . .        |
|=  E.    . .     |
|o.  +o    +      |
|..   ++ S  o     |
|o    .+* ..      |
| . .  o++..o.    |
|  =  o+ +.+. .   |
| o.o.o=*.. ....  |
+----[SHA256]-----+

> cat ~/.ssh/transfer-family-test.pub | pbcopy

enter the public SSH key and click Add.

※ You can follow the instruction here to create an SSH public key.

Confirm that the user is successfully added

Install client application

Install Cyberduck

> brew install cyberduck --cask
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/cask-versions, homebrew/core and homebrew/cask).
==> New Formulae
snowpack
==> Updated Formulae
Updated 28 formulae.
==> New Casks
devbook                                 goldenpassport                          netnewswire-beta
==> Updated Casks
Updated 26 casks.

==> Downloading https://update.cyberduck.io/Cyberduck-7.8.2.34203.zip
==> Downloading from https://a48823c7ec3cf4539564-60c534a1284a12ce74ef84032e9b4e46.ssl.cf1.rackcdn.com/Cyberduck-7.8.2.
######################################################################## 100.0%
==> Installing Cask cyberduck
==> Moving App 'Cyberduck.app' to '/Applications/Cyberduck.app'
🍺  cyberduck was successfully installed!

Set up configurations

If Cyberduck is successfully installed, open the application. Click + button.

Set the following configuration

• SFTP
• endpoint → you can get it on AWS console

Test connection

Check that there is no file on S3

> aws s3 ls s3://transfer-family-test-miyahana-local
# There is nothing because we have never transferred any files

Open directory where the file you want to transfer locates, drag, and drop the file. You can copy the file from local to S3!!!

Check S3

> aws s3 ls s3://transfer-family-test-miyahana-local/
2021-03-09 00:04:58         12 test.txt

> aws s3 cp s3://transfer-family-test-miyahana-local/test.txt .
download: s3://transfer-family-test-miyahana-local/test.txt to ./test.txt

> cat test.txt
hello world

Clean up

• Delete the server
• Delete ssh key pairs
• Delete the bucket

You can follow instructions