video_editing:davinci_resolve_in_the_cloud

DaVinci Resolve in the cloud

I'm trying to keep up with my notes on Video Editing and DaVinci Resolve. Hardware choices for DaVinci resolve is a subject for a small book (which needs updating every 6 months), but I've summarised my notes under DaVinci Resolve Specs.

My questions were:

Can I use cloud computing to run DaVinci Resolve?
At what point does it become cheaper to buy a new laptop for “editing on the go” in comparison to using cloud computing?

This came from the fact that I had an older (but not that old) laptop which could no-longer keep up with running DaVinci resolve (at least reasonably). I didn't need to upgrade it for any productivity outside of running DaVinci Resolve.

Laptop prices showed that a laptop with an Nvidia card that would run DaVinci Resolve confidently for a few years would cost about £3,000 If I went with Apple, then the new M1 chip brings the costs down. The Macbook Air which comes in at just over £1,000 (Or just under £1,500 for a more powerful Macbook Pro with the M1 chip).

After doing some numbers (just for DaVinci Resolve), I've calculated (at least for UK prices) that a £1,000 laptop used 10 days a year over a 5 year lifetime works out at £20 a day. Worst case scenario is a £3,000 laptop used for 5 days a year over a 3 year lifetime, which averages out at £200 a day. That's a lot of variance!

If I can find a way of working which comes to less than £20 ($27.36) a day then I'm on par with my cheapest solution for my predicted usage.

This can be achieved by using one of:

This looks at using powerful windows virtual machines in the cloud, they will run both versions of DaVinci Resolve (Free and Studio). However studio will be able to use more of the fancy GPU features. Warning from personal experience, getting a dongle to work remotely is hard/impossible (I couldn't do it) and I've been unable to test with a licence key (so much for the dongle for when I'm working without internet).

This mostly looks at two EC2 instances:

I work from a PC which has good hardware, and then I have a laptop (which is now getting old) which I use on the go and for work related stuff. The laptop is capable of doing most things I require with the exception that it runs resolve painfully slowly!

Do I:

  • Buy a new laptop with the hardware to run Resolve happily for a few years… In the windows world we are easily talking about £2k or more, more than I want to spend on something I don't use that often.
  • Buy a new M1 chip laptop at around £1k, it's tempting but I don't want to invest so much in hardware.
  • Put up with a laptop which is just frustrating with its workflow.
  • Something else…

I don't tend to use the laptop except on some occasions when I'm on the go, so perhaps 5 days a year worth of time (under COVID we can bump that up to 10 days). For hardware, I'm going to expect it to be happy for 3 years, and usable for 5.

This is what different costs look like. The lowest cost option is if I spend only £1,000 on a laptop and that laptop keeps me happy for 5 years (from 2021 to 2026), and I use that laptop 10 times a year for video editing (yes, I appreciate there will be other advantages to it). But that still lands me at £20 a day and if I actually go to only 5 days a year of usage, that's £40. Switching to only being happy with it for 3 years, that's £33.33 and £66.67.

Cost Years Cost per year Days per year Cost per day
£1,000.00 5 £200.00 10 £20.00
£2,000.00 5 £400.00 10 £40.00
£3,000.00 5 £600.00 10 £60.00
£1,000.00 3 £333.33 10 £33.33
£2,000.00 3 £666.67 10 £66.67
£3,000.00 3 £1,000.00 10 £100.00
£1,000.00 5 £200.00 5 £40.00
£2,000.00 5 £400.00 5 £80.00
£3,000.00 5 £600.00 5 £120.00
£1,000.00 3 £333.33 5 £66.67
£2,000.00 3 £666.67 5 £133.33
£3,000.00 3 £1,000.00 5 £200.00

I work with cloud computing professionally, as well as using DigitalOcean for a virtual machine for my wiki, blog and other projects.

Note: The link above is my Referral link, If you sign up using it we both win! Everyone I refer gets $100 in credit over 60 days. Once they’ve spent $25 with DigitalOcean, I'll get $25 (we all win).

I found this article Creating a feature film with DaVinci Resolve on AWS and was inspired to think differently. While the article focuses on providing a super powerful environment and infrastructure for supporting a large project, the same philosophy can apply for occasional hobby usage.

I also found an article and PDF on using AWS environments for doing 3D rendering and graphics work:

And an article for AWS and Post Production (Adobe CC) products:

I calculated, that if I can find a way of working which comes to less than £20 a day then I'm on par with my cheapest solution for my predicted usage.

AWS is priced in USD, at the time of this note, Xe Currency Converter: 10 GBP to USD = 13.6823 US Dollars. So my aim is to stay below $27.36 USD per day.

To compare, I looked at the EC2 prices based on the Paris region:

NVIDIA Gaming PC - Windows Server 2019:

Model g4dn.xlarge g4dn.2xlarge g4dn.4xlarge
Cores 4 8 16
Memory (GB) 16 32 64
Hour Cost (USD) 0.799 1.247 2.144
8 Hour Day (USD) 6.392 9.976 17.152
10 Hour Day (USD) 7.99 12.47 21.44
12 Hour Day (USD) 9.588 14.964 25.728

Using the 16 core machine for 12 hours (marathon day), I'm keeping (just) below the $27.36/£20 a day price I'm aiming for, although tax and data storage might push it over. So, instead to remain cost effective I'm better either using it for a shorter period of time or dropping down to a smaller machine. Fortunately using cloud resources makes this scaling up and down easy.

DaVinci Resolve 17 Studio:

Model g4dn.xlarge g4dn.2xlarge g4dn.4xlarge g4dn.8xlarge
Cores N/A N/A 16 32
Memory (GB) N/A N/A 64 128
Hour Cost (USD) N/A N/A 1.409 2.546
8 Hour Day (USD) N/A N/A 11.272 20.368
10 Hour Day (USD) N/A N/A 14.09 25.46
12 Hour Day (USD) N/A N/A 16.908 30.552

Using the 16 core machine for 12 hours (marathon day), I'm keeping well below the $27.36/£20 a day price I'm aiming for. I could go up to the more powerful instance for shorter bursts of productivity, if the requirement was there.

When you stop (turn off) the virtual machine, you stop paying the prices above. Instead you only pay for the storage:

When you stop an instance, we shut it down. We don't charge usage for a stopped instance, or data transfer fees, but we do charge for the storage for any Amazon EBS volumes.

At time of writing, the costs for EBS show:

gp2 - 30  GB = $  3.48
gp3 - 200 GB = $ 18.56

By default, the instance configuration contains the 30BG of gp2 (plus a 200GB disk which is thrown out when the machine is off), so it will cost a total of $3.48 for a month of a switched off machine (software installed and configured).

If you want to store more data for longer periods (like over the month) it's better to use S3 (checked against the s3 pricing).

Type: Cost per GB-month GB Monthly Cost
General Purpose SSD (gp2) Volumes $0.1160 30 $3.48
General Purpose SSD (gp3) – Storage $0.0928 30 $2.78
General Purpose SSD (gp3) – Storage $0.0928 200 $18.56
S3 Standard - First 50 TB / Month $0.0240 200 $4.80
S3 Standard - First 50 TB / Month $0.0240 1,000 $24.00

I'm going to focus on two key areas of the AWS catalogue (it's massive, they have offerings and options than the biggest supermarket has breakfast cereals).

  1. EC2 (Elastic Compute Cloud): This is a virtual machine offering, allowing people to create as many virtual computers in the cloud as they like within minuets. Then thrown them away when they have finished with them, only paying for the time that the machine “exists”. Amazon EC2 - Amazon AWS
  2. S3 (Simple Storage Service): This is virtual disk storage, hosted by amazon. It's commonly used for big data solutions, allowing users to create “buckets” to put data into. But it's also a cost effective way of pushing small to extremely large amounts of data into the cloud and into the AWS ecosystem (ready for use by an EC2 instance. Simple Storage Service (Amazon S3) - Amazon AWS

EC2 is the main focus of this workflow, as I'm interested in leveraging the available AWS and NVIDIA virtual machines. DaVinci Resolve leverages the GPU, and having a nice powerful NVIDIA T4 GPU. Note, only DaVinci Resolve Studio (non free version) will fully utilise the GPU (although the free version gets a good portion of it).

I can't be bothered to cover this, it's easy to sign up for an account with amazon for AWS and get that up and running (do allow a little time for it).

To be able to run the G4dn Instances under EC2, I had to extend my quota: Amazon EC2 service quotas

To request an increase using the Amazon EC2 console
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- From the navigation bar, select a Region.
- From the navigation pane, choose Limits.
- Select the resource in the list, and choose Request limit increase.
- Complete the required fields on the limit increase form and choose Submit. We'll respond to you using the contact method that you specified.

Configure Service Quota:

  • Running On-Demand G instances - I set this to 16 so I could use up to (but not accidentally more than) a 16 core machine.

Configure Access Tokens

Before bucket, you will need these in VM. Easy/command line access to S3 bucket.

Tokens:

Create Bucket

Follow S3 instructions

Upload Data

Follow S3 instructions

You will want to create one of these, it's a preconfigured server image for the Nvidia hardware, you can select the number of vCPU cores and amount of RAM:

Select VM type, I recommend filtering for g4dn and selecting either the 8 or 16 core machine (more money = more power).

You will need to get the admin password (wait 4 mins after creation of VM), that required using the key pair.

Once you have the hostname, username and password, you can connect using Windows Remote Desktop (search for mstsc in the start menu)

Note: It does allow you to download a configuration file with the username and hostname which you can use to launch the application rather than copy and paste into the application.

If you are reconnecting:

  1. Check the firewall rules, by default the setup uses the IP address from which you first connected. So, if you connect from a new outside location (or your home IP changes) the firewall will block you.
  2. Check the Public DNS. If you stop, then restart the EC2 instance, the IP address (and Public DNS associated with it) will change.

If you get too many old connections:

DaVinci Resolve fails to install (thanks to the workings of default PostgreSQL installation). As this isn't a production world facing server, we can cut back the security.

  1. Local Security Policy (Editor)
    1. Account Policies → Password Policy: Password must meet complexity requirements = Disabled
  2. Local Group Policy Editor
    1. Computer Configuration → Administrative Templates → Windows Components → Internet Explorer → Internet Control Panel → Advanced Page → Turn On Enhanced Protected Mode = Disabled

Other Configuration steps

  1. Format/create hard disk
    1. Note: If you “stop” the VM to reconnect again, the disk D will be gone
  2. Enable “Windows Audio” service (set to start on startup)

Software installation:

  1. AWS CLI (for S3)
  2. ninite.com Notepad++, peazip, VLC

Optional Session Manager

Note: If you “stop” the VM to reconnect again, the disk D will be gone

Copy vir RDP…

or, using S3

https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-access-s3-bucket/

AWS cli, copy files to/from S3

Using access keys from above

aws configure
aws s3 ls s3://<name>

aws s3 ls s3://drh-video-demo-1

D:\>aws s3 cp s3://<name>/ D:\ --recursive

D:\>aws s3 cp s3://drh-video-demo-1/ D:\ --recursive

You should now be able to use the VM via remote desktop like any normal machine.

D:\>aws s3 cp <local directory> s3://<name>/<name> --recursive

D:\>aws s3 cp dump s3://drh-video-demo-1/d1 --recursive

This is super important as you pay for the machine all the time you are using it.

AWS DaVinci Resolve Workflow Reconnect

Use a bat script to opy the files from local machine to s3.

push_projects_to_aws.bat
@ECHO OFF
REM ===========================================================================
REM Set common variables
REM project_list will be whatever directories you wish to copy from the s3 projects directory to local
REM ===========================================================================
SET aws_s3_bucket=drh-video1
SET drive_location="C:\Users\david\Pictures\Adventures In Cloud Computing"
 
SET project_list=dir_name_1 dir_name_2
 
REM remove quotes
SET drive_location=%drive_location:"=%
 
REM ===========================================================================
REM Set varialbes used in script
REM ===========================================================================
 
REM SET local_project_location=%drive_location%\projects
SET local_project_location=%drive_location%
SET s3_project_location=s3://%aws_s3_bucket%/projects
 
echo Local project location: %local_project_location%
echo    S3 project location: %s3_project_location%
 
REM ===========================================================================
REM Push projects up
REM ===========================================================================
 
FOR %%i in (%project_list%) do (
echo Project: %%i
aws s3 sync "%local_project_location%/%%i" %s3_project_location%/%%i
)
 
pause

Remember before connecting:

  1. Check new IP/DNS Name

The second disk isn't kept, but can be recreated using a script

After connecting:

script_diskpart.txt
select disk 1
clean
create partition primary
format quick
assign letter=D
d_drive_format.bat
diskpart /s script_diskpart.txt
pause
copy_to_d.bat
@ECHO OFF
REM ===========================================================================
REM Set common variables
REM project_list will be whatever directories you wish to copy from the s3 projects directory to local
REM ===========================================================================
SET aws_s3_bucket=drh-video1
SET drive_location=D:\projects
REM SET drive_location=C:\Users\david\projects
 
SET project_list=OVFM logo
 
REM ===========================================================================
REM Set varialbes used in script
REM ===========================================================================
 
SET local_project_location=%drive_location%
SET s3_project_location=s3://%aws_s3_bucket%/projects
 
echo Local project location: %local_project_location%
echo    S3 project location: %s3_project_location%
 
REM ===========================================================================
REM List available projects
REM ===========================================================================
echo ===========================================================================
echo Available Projects
echo ===========================================================================
aws s3 ls %s3_project_location%/
 
REM ===========================================================================
REM Create some local directories
REM ===========================================================================
 
mkdir %drive_location%
mkdir D:\MEDIA
 
REM ===========================================================================
REM Pull projects down
REM ===========================================================================
echo ===========================================================================
echo Downloading Configured Projects
echo ===========================================================================
 
FOR %%i in (%project_list%) do (
aws s3 cp %s3_project_location%/%%i %local_project_location%\%%i --recursive
)
 
pause
sync_to_s3.bat
@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
REM ===========================================================================
REM Set common variables
REM ===========================================================================
SET aws_s3_bucket=drh-video1
SET drive_location=D:\projects
 
REM ===========================================================================
REM Set varialbes used in script
REM ===========================================================================
 
SET local_project_location=%drive_location%
SET s3_project_location=s3://%aws_s3_bucket%/projects
 
echo Local project location: %local_project_location%
echo    S3 project location: %s3_project_location%
 
REM ===========================================================================
REM Loop directories in projects directory and sync to s3
REM ===========================================================================
for /d %%I in (%local_project_location%\*) do (
SET FullDirName=%%I
SET CurrDirName=%%~nxI
echo Project Name: !CurrDirName!
echo Project Path: !FullDirName!
aws s3 sync !FullDirName! %s3_project_location%/!CurrDirName!
)
 
pause
  • video_editing/davinci_resolve_in_the_cloud.txt
  • Last modified: 2021/11/22 09:55
  • by david