Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

batchiepatchie backend crashes when new job spins up #93

Open
erhaben-Raupe opened this issue Jan 19, 2023 · 0 comments
Open

batchiepatchie backend crashes when new job spins up #93

erhaben-Raupe opened this issue Jan 19, 2023 · 0 comments

Comments

@erhaben-Raupe
Copy link

erhaben-Raupe commented Jan 19, 2023

We have batchie patchie running in a production setup. Whenever new job spins up backend throws nil pointer reference and crashes.

{"level":"info","msg":"Logging ECS cluster statuses.","time":"2023-01-19T11:25:36Z"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa54007]

goroutine 11 [running]:
github.com/AdRoll/batchiepatchie/jobs.MonitorECSClusters({0x7f192019aff8, 0xc000012c60}, {0x160a640, 0x0, 0x1?})
	/home/ubuntu/batchiepatchie/jobs/monitor_ecs_clusters.go:126 +0x16e7
github.com/AdRoll/batchiepatchie/syncer.RunPeriodicScaler.func1()
	/home/ubuntu/batchiepatchie/syncer/batchsync.go:296 +0x13e
created by github.com/AdRoll/batchiepatchie/syncer.RunPeriodicScaler
	/home/ubuntu/batchiepatchie/syncer/batchsync.go:287 +0x6a

This happens when the job status is STARTING in AWS console. When the status change to RUNNING I can restart batchiepatchie again.

FYI, my setup is running on ubuntu 22.04 in an EC2 and the capacity provider for batch is Fargate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant