Lambda function scheduled to perform backups of another server on s3
Ep.3 - CDK first steps
15 minutes
We will run cdk init to start a Typescript project.
• Updating our Makefile to document the setup commands
• Creating a folder with the stack, service, and dependencies
• Install missing aws and system dependencies as we need them
We start by defining a new s3 bucket we want to create.
Notes
• Source code in Githubhttps://github.com/fromzerotofullstack/lambda-cron-backup-s3/tree/
• Amazon s3 Bucket docs
https://docs.aws.amazon.com/cdk/api/latest/typescript/api/aws-s3/bucket.html
• CDK hello worls
https://docs.aws.amazon.com/cdk/latest/guide/hello_world.html
Transcript
now that we install the cdk we can start with our project so first of all let's
let's check that the cdk is actually installed yes
so here's the the command with all the options
so this is correct and then we're ready to
to uh initialize the project so we're going to create a folder
and because lambda likes to have like a folder so we're going to
call this folder chrome backup s3
and then we can just go inside right and then here we're going to
initialize our project so that our project will be contained in this
folder that is easier later to deploy um we can deploy a whole folder
so that's why we're doing this and we're gonna add also these steps
here in the make file so to do this because this makefile runs from the root
we're gonna do like a chrome backup s3
and then the actual command cdk in it and in our case we want
typescript as language so here you could use python javascript
javascript i think by default it's javascript
but we don't type script so this we can even
either write run the make file in the from the root folder
or from this chrome backup and which is where we are now and then
we just run this command so let's run this and this is going to
create like a project with all the the defaults right and then we'll be
able to run this cdk deploy cdk div all these commands
um to deploy our resources to like destroy them so will go
step by step through each of these but first of all
this has um this is updated it recommends a new
version of the cdk we already run this yes we already run
it so we can run it again it is a new version
okay and this never okay so now we got the latest version
and if we look at the files we should have
a basic project here so we can look at it here
we see we have like package with some basic comments
dependencies we have a typescript config also
and there's a readme yeah so these are some of the comments we can use
and yeah basically an empty empty project with some library
files and here is where we're gonna start
using our stuff so here in this library yeah so we'll see we'll see in a second
how to how to update anything but with this you
should have the the basic file created so we can see here uh there's already a
comment here right the code that defines your stack goes here
so what is this this is basically the code we will write
that will say what stack we want to allocate
so we're using the cdk right the cloud development kit so we're defining with
code the resources we want so um
you could just put the code here but i think a slightly cleaner way
is to um okay a slightly cleaner way is it to
call uh to use this service uh way so basically we'll define this in
another file and this file is just gonna call that service
so we're gonna call that widget service and it will be called chrome backup
s3 service all of this doesn't exist yet so that's why we are seeing it in red
and here we will pass an id let's record backup
okay so this is something and um we'll need to import this as well so now
we'll define it but basically we'll import um
as widget service the file where the file in
like outside of library lib we can also just refer to the same
location but that's where this um chrome backup
has three services and this is the file that we're gonna need right
so this file we're gonna create right now
so just here is gonna be a typescript file
here and now we have the file here and you
want to add the following no we cannot took it manually
um okay so now this is loading this chrome backup services from here
right so we're gonna need a few imports here
and basically all this boilerplate right so it's going to be a class called
chromebackup s3 service that extends
okay um i'm wondering i think for now
maybe the idea is not getting this private name core
maybe we're missing some imports but uh let's try to write the theme
so um here we will define all our resources right for this service for the
chrome backup so we could have here several services
each with its own definition uh we will fix this uh all of these errors
or maybe they they'll get fixed by themselves one day
once we implement this and but basically here there's still
some more boiler plate construct uh and then id these are all type
annotations for typescript right so what we are writing here is
typescript and let's go to 90. okay so more or less
this is the this is the boilerplate and actually
here we put our code right here so um all of this boilerplate
is to start defining resources we will need
right so our resource we're gonna need is a s3
so like storage so we want i want to run this chrome
and then save something in some storage right say
a backup so we're gonna start by by dealing with the storage
so to do that i'm going to create here a bucket
now we'll use s3 s3 has this bucket
method in the api then i'm gonna assign the name from backup s3 just an
id okay so it will be something like this
and here there's all the options so we can have like i can i can
i can show you so basically you can go here search like
s3 and you can see here all the
documentation including the the bucket right and for language
so in this case uh typescript and we can even go to
to the bucket one right or like here i mean they're quite easy
to navigate and here you have all the options like how to create
uh for instance if it's versioned or not [Music]
[Music] yeah like names here um
yeah so we got we can use like all these options right so um we're
gonna start uh adding some of these with things we want
like for instance if the name is the back of this version and set this
to false then the bucket name
and we're gonna call it chrome backup s3 then the public fit
access folds because we don't want this to be
public and this this other um
parameter got public access i'm just going to set that to s3
block and access
so there's many ways to do this but this is the one we're going to use
and there's a lot of options here so you know some of these very could be
accomplished in using other properties but so far
we're going to start with this so this code that we are writing should
create a bucket right so far uh so good and but first let's
fix all these problems that are cropping up
like we're missing some imports here for sure
um like s3 is not imported so let's start with that
uh here we want to import everything from s3
or six three from and then this we can check in the
documentation but basically see the ks3 right this is the url
so we can just import this [Music]
okay so from here we're importing it and i'm sure we're missing more um
yeah let's start with this this one got soft the
core one didn't get soft yet
this one is looking good also let's see we're gonna need to install a
few more things this one disappear that now that is implemented
yes this one still cannot define namespace
core uh here there's a mistake there's no cdk but
okay so now this makes a lot more sense and i think this one is not installed
right so we can go back to the make file
and install it so we're gonna copy it and then here
[Music] basically we can see exactly all the
installed so there's one thing here that is
interesting that basically we installed here in the global
um so for every user in our computer and every path
this uh cdk right but the dependencies that the
this thing has need to be defined inside the project
inside this chrome backup s3 so here there's a node modules as you can see
and this node modules is the one that's going to be shipped to our lambda
so this is what needs to have the dependencies right
so this means that instead of adding this here
in the install like this we're gonna add it
inside the folder so cv chrome backup and then here we're going to install it
so like npm [Music]
npm install and like this because this will
instead inside the node module so we can we can
check that if we go to node modules uh we can see here aws cdk right
but not not this one right so uh let's see
so let's install it and see if that changes and it should
um so basically as always we can run this
this make file actually let's run it we can even either run this command from
the folder where we are the chrome backup right or we can just
call backup one folder and run make local init
[Applause] [Music]
so we can just do that just to show that both work make local init
make a command not found okay we don't have
make file installed here let's quickly install the make file too
just to show you that that this can run so let's just make i'm gonna add it here
as a comment i usually make a spring instead we're
running this from a docker image so that's why it cannot be found but not a
problem we'll install it perfectly installed and then
here and now we have make so now we can run our command
local image he's running all the commands but in the
end it's also installing everything right
uh-huh see the init cannot be run in a non-empty
directory to be already around right so we don't
need that one um
[Music] exactly so basically this initial um
maybe we can call this like um we could recommend this out
it is called like local install i guess and then we just call that
so make a local because we already initialized it right
so cdk is not letting us initialize it could also ignore it but okay so now it
installed it so now we have this uh amazon web services s3 and when we check
in in our service this warning has disappeared already
so this is looking good to and it's looking ready to
to be run so we're gonna we're gonna run it and see if we're
missing something
Clone the repository
git clone https://github.com/fromzerotofullstack/lambda-cron-backup-s3
Episodes
Ep.1 - Introduction: Lambda cron backup to s3
(4 minutes)
We explain how the project will be structured and what we are trying to accomplish: a cron process running on AWS Lambda to backup an external server.
• AWS Lambda
• AWS Cloudformation
• AWS CDK
• AWS S3
Ep.2 - Installing dependencies
(6 minutes)
We start installing the dependencies for the project.
• curl
• Nodejs and npm
• AWS CDK
We will start organizing the commands used to install dependencies on a Makefile
Ep.3 - CDK first steps
(15 minutes)
We will run cdk init to start a Typescript project.
• Updating our Makefile to document the setup commands
• Creating a folder with the stack, service, and dependencies
• Install missing aws and system dependencies as we need them
We start by defining a new s3 bucket we want to create.
Ep.4 - CDK deploy and destroy
(26 minutes)
We will deploy our resources with cdk deploy. Once we are done we will use cdk destroy to remove the resources.
• Updating our Makefile to document the changes.
• Confirm the s3 bucket is created.
• Deal with common pitfalls when running cdk destroy — like s3 resoruces not being removed.
• Install autodelete bucket and missing aws dependencies.
• Makefile .PHONY targets.
• Permissions.
We will allocate and deallocate aws resources using cdk deploy and cdk destroy, and deal with some common problems we can run into.