Skip to content

Commit

Permalink
Merge pull request #160 from aalug/develop
Browse files Browse the repository at this point in the history
add util functions to load sample data (#159)
  • Loading branch information
aalug committed Aug 27, 2023
2 parents ee93353 + 95fdf01 commit b179035
Show file tree
Hide file tree
Showing 9 changed files with 192 additions and 13 deletions.
13 changes: 5 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,17 @@ test:
test_coverage:
go test $(p) -coverprofile=coverage.out && go tool cover -html=coverage.out

# run main function -> start the server.
# $(load_data): boolean - set true if you want to load test/ sample data into the postgres and elasticsearch
runserver:
go run cmd/main.go
go run cmd/main.go -load_test_data=$(load_data)

# flush db and restart it
flush_db:
# flush containers and restart it
flush:
docker-compose down
docker volume ls -qf dangling=true | xargs docker volume rm
docker-compose up -d

flush_es:
docker-compose stop elasticsearch
docker-compose rm -f elasticsearch
docker-compose up -d elasticsearch

# generate swag documentation files
swag:
swag init -g cmd/main.go
Expand Down
25 changes: 20 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,15 @@ account types have access to different endpoints and are allowed to perform
different actions (e.g. only employers can create job offers, and only users
can create job applications).
Search functionality is implemented both with Postgres and Elasticsearch (depending on search complexity).
Redis is used to perform tasks in the background such as sending verification or confirmation emails.
<hr>

### Built in Go 1.20

### The app uses:
- Postgres
- Redis
- Docker
- Redis
- [Elasticsearch](https://github.com/elastic/go-elasticsearch)
- [Gin](https://github.com/gin-gonic/gin)
- [golang-migrate](https://github.com/golang-migrate/migrate)
Expand All @@ -30,11 +31,13 @@ Search functionality is implemented both with Postgres and Elasticsearch (depend
1. Clone the repository
2. Go to the project's root directory
3. Rename `app.env.example` to `app.env` and replace the values
4. Run in your terminal:
- `docker-compose up` to run the database container
4. Install [golang-migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate)
5. Run in your terminal:
- `docker-compose up` - to run the containers
- `make migrate_up` - to run migrations
- `make runserver` - to run HTTP server
5. Now everything should be ready and server running on `SERVER_ADDRESS` specified in `app.env`
- `make runserver load_data=true` - to run HTTP server with loaded (into the postgres and elasticsearch)
sample jobs, employers and companies (set `load_data` to `false` if you do not want test data to load)
6. Now everything should be ready and server running on `SERVER_ADDRESS` specified in `app.env`
<hr>

## Testing
Expand Down Expand Up @@ -88,6 +91,12 @@ If a user with the given email already exists, a `403 Forbidden` status code is
of any other error, a `500 Internal Server Error` status code is returned.
After registering, a verification email is sent to the provided email address.

+ `GET /users/send-verification-email`: This endpoint sends an email to the user with a link that should be used
to verify their email address. The request must contain the user’s email as a query parameter. On success, the response
has a `200 OK` status code and returns the result in JSON format. If the email is invalid, a `400 Bad Request` status code
is returned. If no user is found with the provided email, a `404 Not Found` status code is returned. In case of any
other error, a `500 Internal Server Error` status code is returned.

+ `GET /users/verify-email`: This endpoint verifies a user’s email by providing a verify email ID and
secret code that should be sent to the user in the verification email. The request body must contain the verify
email ID and secret code as query parameters. On success, the response has a `200 OK` status code and returns the
Expand Down Expand Up @@ -144,6 +153,12 @@ employer with the given email already exists, a `403 Forbidden` status
code is returned. In case of any other error, a 500 Internal Error status code is returned.
After registering, a verification email is sent to the provided email address.

+ `GET /employers/send-verification-email`: This endpoint sends an email to the employer with a link that should be used
to verify their email address. The request must contain the employer’s email as a query parameter. On success, the response
has a `200 OK` status code and returns the result in JSON format. If the email is invalid, a `400 Bad Request` status code
is returned. If no employer is found with the provided email, a `404 Not Found` status code is returned. In case of any
other error, a `500 Internal Server Error` status code is returned.

+ `GET /employers/verify-email`: This endpoint verifies an employer’s email by providing a verify email ID and
secret code that should be sent to the user in the verification email. The request body must contain the verify
email ID and secret code as query parameters. On success, the response has a `200 OK` status code and returns the
Expand Down
10 changes: 10 additions & 0 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package main
import (
"context"
"database/sql"
"flag"
"github.com/aalug/go-gin-job-search/internal/api"
"github.com/aalug/go-gin-job-search/internal/config"
"github.com/aalug/go-gin-job-search/internal/db/sqlc"
Expand Down Expand Up @@ -35,6 +36,15 @@ func main() {

taskDistributor := worker.NewRedisTaskDistributor(redisOpt)

// === loading test data ===
loadDataFlag := flag.Bool("load_test_data", false, "If set, the application will load test data into db")
flag.Parse()

if *loadDataFlag != false {
// load the test data into the db
store.LoadTestData(context.Background())
}

// === Elasticsearch ===
ctx := context.Background()
ctx, err = esearch.LoadJobsFromDB(ctx, store)
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ require (
github.com/aead/chacha20 v0.0.0-20180709150244-8b13a72661da // indirect
github.com/aead/chacha20poly1305 v0.0.0-20170617001512-233f39982aeb // indirect
github.com/aead/poly1305 v0.0.0-20180717145839-3fee0db0b635 // indirect
github.com/bxcodec/faker/v3 v3.8.1 // indirect
github.com/bytedance/sonic v1.10.0-rc2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ github.com/bsm/ginkgo/v2 v2.7.0 h1:ItPMPH90RbmZJt5GtkcNvIRuGEdwlBItdNVoyzaNQao=
github.com/bsm/ginkgo/v2 v2.7.0/go.mod h1:AiKlXPm7ItEHNc/2+OkrNG4E0ITzojb9/xWzvQ9XZ9w=
github.com/bsm/gomega v1.26.0 h1:LhQm+AFcgV2M0WyKroMASzAzCAJVpAxQXv4SaI9a69Y=
github.com/bsm/gomega v1.26.0/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/bxcodec/faker/v3 v3.8.1 h1:qO/Xq19V6uHt2xujwpaetgKhraGCapqY2CRWGD/SqcM=
github.com/bxcodec/faker/v3 v3.8.1/go.mod h1:DdSDccxF5msjFo5aO4vrobRQ8nIApg8kq3QWPEQD6+o=
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
github.com/bytedance/sonic v1.10.0-rc/go.mod h1:ElCzW+ufi8qKqNW0FY314xriJhyJhuoJ3gFZdAHF7NM=
github.com/bytedance/sonic v1.10.0-rc2 h1:oDfRZ+4m6AYCOC0GFeOCeYqvBmucy1isvouS2K0cPzo=
Expand Down
12 changes: 12 additions & 0 deletions internal/db/mock/store.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

82 changes: 82 additions & 0 deletions internal/db/sqlc/load_test_data.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package db

import (
"context"
"github.com/aalug/go-gin-job-search/pkg/utils"
"github.com/bxcodec/faker/v3"
"log"
"sync"
"sync/atomic"
)

// LoadTestData loads the test data into the database
func (store *SQLStore) LoadTestData(ctx context.Context) {
var wg sync.WaitGroup
nOfJobsCreated := int32(0)
jobTitles := append(utils.GenerateEngineerJobs(), utils.GenerateDeveloperJobs()...)

// create fake companies
for i := 0; i < 5; i++ {
for _, industry := range utils.Industries {
// Increment the WaitGroup counter for each goroutine
wg.Add(1)

go func(industry string) {
// Decrement the WaitGroup counter when the goroutine is done
defer wg.Done()

idx := utils.RandomInt(0, int32(len(utils.Locations)-1))
location := utils.Locations[idx]
companyParams := CreateCompanyParams{
Name: faker.DomainName(),
Industry: industry,
Location: location,
}
company, err := store.CreateCompany(ctx, companyParams)
if err != nil {
log.Println(err)
return
}

var jobsCreated int32
var jobWg sync.WaitGroup

// create jobs
for j := 0; j < 3; j++ {
// Increment the job WaitGroup counter
jobWg.Add(1)

go func() {
// Decrement the job WaitGroup counter when the job is done
defer jobWg.Done()

idx := utils.RandomInt(0, int32(len(jobTitles)-1))
jobTitle := jobTitles[idx]
jobParams := CreateJobParams{
Title: jobTitle,
Industry: industry,
CompanyID: company.ID,
Description: jobTitle + " " + faker.Paragraph(),
Location: location,
SalaryMin: utils.RandomInt(0, 2000),
SalaryMax: utils.RandomInt(2001, 5000),
Requirements: jobTitle + " " + faker.Paragraph(),
}
_, err := store.CreateJob(ctx, jobParams)
if err != nil {
log.Println(err)
} else {
atomic.AddInt32(&jobsCreated, 1)
}
}()
}

jobWg.Wait() // Wait for all jobs to finish
atomic.AddInt32(&nOfJobsCreated, jobsCreated)
}(industry)
}
}

wg.Wait() // Wait for all companies to finish
log.Printf("Created %d jobs", nOfJobsCreated)
}
1 change: 1 addition & 0 deletions internal/db/sqlc/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ type Store interface {
VerifyUserEmailTx(ctx context.Context, arg VerifyEmailTxParams) (VerifyUserEmailResult, error)
VerifyEmployerEmailTx(ctx context.Context, arg VerifyEmailTxParams) (VerifyEmployerEmailResult, error)
CreateJobApplicationTx(ctx context.Context, arg CreateJobApplicationTxParams) (CreateJobApplicationTxResult, error)
LoadTestData(ctx context.Context)
}

// SQLStore provides all functions to execute db queries and transactions
Expand Down
59 changes: 59 additions & 0 deletions pkg/utils/generate_db_data.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package utils

import "fmt"

var Industries = []string{
"IT",
"Game Development",
"Financial Technology",
"E-commerce",
}

var Locations = []string{
"London",
"New York",
"Paris",
"Berlin",
"Madrid",
"Milan",
"Rome",
"Amsterdam",
"Barcelona",
"Vienna",
"Hamburg",
"Dublin",
"Sydney",
"Singapore",
"Hong Kong",
"Tokyo",
"Seoul",
"Warsaw",
}

// qualifiers
var qualifiers = []string{"Junior", "Senior", "Lead"}

// languages for Developers and Engineers
var languages = []string{"Java", "Python", "JavaScript", "Go", "C#", "C++", "Ruby", "C"}

// GenerateDeveloperJobs Function to combine different parts for Developers
func GenerateDeveloperJobs() []string {
var combined []string
for _, qualifier := range qualifiers {
for _, language := range languages {
combined = append(combined, fmt.Sprintf("%s %s Developer", qualifier, language))
}
}
return combined
}

// GenerateEngineerJobs Function to combine different parts for Engineers
func GenerateEngineerJobs() []string {
var combined []string
for _, qualifier := range qualifiers {
for _, language := range languages {
combined = append(combined, fmt.Sprintf("%s %s Engineer", qualifier, language))
}
}
return combined
}

0 comments on commit b179035

Please sign in to comment.