Adding support for EFS mounts.

Changelog:
* Upgraded CDK version to support EFS usage
* Upgraded Fargate PlatformVersion to support EFS mounts
* Refacored RDS contruct as per new CDK
* Created a new LogGroup for OnDemand DagTasks
* Added TAG for stack, to track resources belonging to this setup
* Updated sample DAG to utilize EFS. Tasks Odd and Even will publish to EFS and Numbers will read from EFS
* Now you can see logs from OnDemand tasks on Airflow UI, once task run finishes
This commit is contained in:
Chaithanya Maisagoni
2020-12-08 11:58:26 -08:00
parent 91438606d9
commit 7814377342
12 changed files with 2568 additions and 576 deletions

View File

@@ -8,6 +8,9 @@ if __name__ == '__main__':
number = args.number
print("Printing Even numbers in given range")
f = open("/shared-volume/even.txt", "a")
for i in range(int(number)):
if(i % 2 == 0):
f.write(str(i))
print(i)
f.close()

View File

@@ -8,6 +8,9 @@ if __name__ == '__main__':
number = args.number
print("Printing Odd numbers in given range")
f = open("/shared-volume/odd.txt", "a")
for i in range(int(number)):
if(i % 2 != 0):
f.write(str(i))
print(i)
f.close()