Standards:
A default exception view - Any exception on any page must take the user to a custom error page if the exception is not handled.
The requests should be accepted only from whitelisted domains. If the referer is not available or if the referer is not in the whitelist, then an error message will be displayed.
---------------------------------------------
REST:
All REST URLs should be using HTTP POST from security perspective.
No input parameters must be present in the PATH variables - the URL/URI should be the generic representation of the resource. The important thing to consider for deciding the URI is the hierarchy of the resource and then the input parameters.
One URI must represent only one resource at the server side. One REST URL is pertaining to one URI. Do not use the same REST endpoint for diffent data.
For example, the postitions URI would be /holdings/positions/previousclose?account={account} rather than /holdings/positions/{account}/previousclose or /holdings/positions/previousclose/{account}. It can also be /holdings/positions?quotes=previousclose&account={account} if the same resource will handle both previous day and realtime dat.
In most cases, the resources respond with a collection. So use plurals in the URI. Eg: holdings/positions in the above URL.
Typically create two URLs (endpoints) per resource:
a. The resource collection (e.g. /holdings/positions - that returns a collection )
b. Individual resource within the collection (e.g. /holdings/positions/detail - that returns detail for one record )
Could respond with JSON/HTML/XML/Excel/PDF/Image content. The content type should be controlled by the HTTP Accept headers preferably or by parameters, not by different controller methods. Use content negotiators for producing different response types.
Use HTTP Error Codes in the response to communicate any system errors to the clients (Browsers) like resource not available, service down, etc and use the status/message attributes to present a meaningful message to the users.
Do not use nested objects in the response - the only nested(inner) object should be a list that contains the actual response. If you want to maintain OO structure / reuse the same classes, the use the mechanisms supported by the JSON libraries to avoid the nested objects in the response. For example, Jackson library offers an annotation called UNWRAP for flatening the inner objects.
Keep URLs as short as possible. Preferably, no more-than three nodes per URL.
Use nouns as resource names - do not use verbs. For example, process / getAccounts / getUsers / etc should not be used for READ transactions.
For any update opertions, use only POST. Do not user PUT/DELETE - thought REST spec suggests these, these should be disabled at the WebServer layer to mitigate any security risks.
Always send the charset code in the response.
Implement Access-Control-Allow-Origin config at the web server level.
Support a Maximum of 200 records per response. If possible, build support for Range/Content-Range HTTP headers to support better pagination.
Use the "Accept" header to support different versions of same resource.
For example, to retrieve a user in JSON format:
# Request
GET http://api.example.com/users/12345
Accept: application/json; version=1
# Response
HTTP/1.1 200 OK
Content-Type: application/json; version=1
{“id”:”12345”, “name”:”Joe DiMaggio”}
Now, to retrieve version 2 of that same resource in JSON format:
# Request
GET http :// api . example . com / users /12345
Accept: application/json; version=2
# Response
HTTP/1.1 200 OK
Content-Type: application/json; version=2
{“id”:”12345”, “firstName”:”Joe”, “lastName”:”DiMaggio”}
The requests should be accepted only from whitelisted domains. If the referer is not available or if the referer is not in the whitelist, then an error message will be displayed.
Use the below to disable caching and set these through a common utility function from the class like ResponseUtil.
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Jan 2015 00:00:00 GMT
E-Tag: hashofthetimestamp
Use Spring REST Docs to document the REST API.
Following are the different entities in the response.
•code – contains the HTTP response status code as an integer.
•status – contains the text: “success”, “fail”, or “error”. Where “fail” is for HTTP status response values from 500-599, “error” is for statuses 400-499, and “success” is for everything else (e.g. 1XX, 2XX and 3XX responses).
•message – only used for “fail” and “error” statuses to contain the error message. For internationalization (i18n) purposes, this could contain a message number or code, either alone or contained within delimiters.
•data – that contains the response body. In the case of “error” or “fail” statuses, this contains the cause, or exception name.
When Should I Create a New Version?
In API development there are many ways to break a contract and negatively impact your clients. If you
are uncertain of the consequences of your change it is better to play it safe and consider versioning.
There are several factors to consider when you are trying to decide if a new version is appropriate or if
a modification of an existing representation is sufficient and acceptable.
Changes that will break contracts
● Changing a property name (ie. “name” to “firstName”)
● Removal of property
● Changing property data type (numeric to string, boolean to bit/numeric, string to datetime, etc.)
● Validation rule change
● In Atom style links, modifying the “rel” value.
● A required resource is being introduced into an existing workflow
● Resource concept/intent change; the concept/intent or the meaning of the resource’s state has a different meaning from it’s original.
Examples:
○ A resource with the content type text/html once meant that the representation would be a collection of “links” to all supported media types, new text/html representation means “web browser form” for user input
○ An API populating an “endTime” on the resource “.../users/{id}/exams/{id}” once meant the student submitted the exam at that time, the new meaning is that it will be the scheduled end time of the exam.
● Adding new fields that came from an existing resource with the intent to deprecate the existing resource. Combining two resources into one and deprecating the two original resources.
○ There are two resources, “.../users/{id}/dropboxBaskets/{id}/messages/{id}” and “.../users/{id}/dropboxBaskets/{id}/messages/{id}/readStatus”. The new requirement is to put the properties from the readStatus resource into the individual message resource and deprecate the readStatus resource. This will cause the removal of a link to the
readStatus resource in the individual messages resource.
While this list is not full-inclusive, it gives you an idea of the types of changes that will cause havoc for your clients and require a new resource or a new version.
https://github.com/tfredrich/RestApiTutorial.com/raw/master/media/RESTful%20Best%20Practices-v1_2.pdf
---------------------------------------------
REST Utility
- read URL from a config (URLName:URL)
- Check for entitlements ??
- pack cookies (pick cookies to be packed from a config entry as diff apps may need diff cookies)
- timeout (configurable request timeout values per URL)
- Common error codes/messages with a facility to override from the consumer
- pagination
- scrollkeys should be passed to browser and then send back, the utility should handle packing/unpacking of scrollkeys
- have a standard name and pack/unpack autimatically based on the name / use an annotation
- logging
- audit
- error
- exception
- nppi masking
- security
- filter invalid characters of request
- perform output encoding of response
RestUtilConfig Sample:
URL=TARGET_URL:www.test.com|ROLE=000 - indicates do not check|CookiesToPack=abc.authservice,site|TO=30000(in milliseconds)|InputFilter=None|OutputEncode=Yes
Create a JS Utility that will help in packing the scrollkeys from browser in each request. Create it as a DOJO addon for the stores.
---------------------------------------------
Question:
Can we enforce a schema for JSON response?
---------------------------------------------
Questions:
How to set view names / URL aliases in Spring?
Spring MVC Training:
@Controller
@Service
@ModelAttribute
@RequestMapping
@ResponseBody
produces
ResponseEntity
HttpHeaders
Interceptors
AOP
Bean Validation:
bean{
@NotNull
@Max(5)
private Integer number;
@NotNull
@Future
@DateTimeFormat(iso=ISO.DATE)
private Date date;
}
@ExceptionHandler
@ControllerAdvice
@PostConstruct
@PreDestroy
Training:
DOJO
HTML 5
CSS 3
Responsive Web Design
Application Security
JVM Optimization
Java Performance Tuning
JUnit
Parameterized JUnit Testing:
http://examples.javacodegeeks.com/core-java/junit/junit-parameterized-test-example/
Selenium
JDBC
Dynatrace
BXP Operational activities
- ID Creation
- VM Creation
- Other VM related activities
- Release Baseline creation
- Code Deployment
- Create a wrapper script - do not have multiple steps/instructions - see if a UI can be created.
logback
Test driven development
-------------------------------------------------------------
Tips:
Spring Boot hosts an Initializr application at http://start.spring.io. The Initializr provides a Web
interface that allows you to enter project information, pick the capabilities needed for your project, and
voilà—it generates the project as a zip file.
-------------------------------------------------------------
A 2002 study by the Carnegie Mellon Software Engineering Institute (SEI) lists the following top 10 reasons why software projects fail:
• Inexperienced staff
• Lack of team cohesion and experience
• Lack of emphasis in using modern software-engineering practices
• Lack of a process or incorrect/insufficient emphasis in the application of a process
• Inadequate project management methodology
• Unclear, misunderstood, and undiscovered requirements
• Size (the larger the projects the more likely they are to fail)
• Lack of planning and estimating
• System-specific and technology-related issues being considered too late in the process
• New technologies and unforeseen problems
-------------------------------------------------------------
Not reading outside work
- If you dont read, you rely on others who may or may not have the expertise and most often you will end up with non-optimal solutions.
- If you read while executing projects, it might delay the execution, some time, you may want to redo the stuff, but will not be allowed as you will not have the time and budget.
-
-------------------------------------------------------------
Testing Best Practices
TDD advocates the practice of programming by writing tests first, writing the code to make the tests pass, then refactoring to optimize the code and the verall design of the application without breaking the tests. Obviously this leads to a comprehensive test suite that covers what your code does, because it was the contract of what was to be built.
1. Write tests first: Write tests that show the typical usage of a given class first (plus some boundary or edge cases). Then write enough code to pass the tests. Coding to interfaces as seen throughout the book can help make the test-first approach attainable.
2. Write code with clear intentions: Follow good object-oriented practices by creating code that makes its purpose obvious to others. Keep classes small and use good naming conventions. It is easier to write a test for a component for which the intent is known, especially since it is possible that somebody else might be writing the tests or enhancing them in the future. This practice becomes easier if you choose to use practice 1.
3. Test everything: If it can break, test it. If it can produce effects or output that’s not clear from its interface, test it. Even simple components can produce errors in other components if their intent or function is not clear. I equate tests in software to formal proofs in mathematics. Unless there is a comprehensive set of tests, any assurance of a component quality is just an illusion.
4. Establish a test harness early in the development: Set up testing as part of the build and if possible, use a test framework that integrates well with your development tools; doing this early in a project will ensure that the whole team embraces testing and that it becomes the path of least resistance.
5. Code enough to pass the unit tests: The task is complete when the unit test is complete. Just like code shouldn’t be checked into a version control system without compiling, code shouldn’t be checked in unless it has a comprehensive set of unit tests and those tests pass.
6. Map tests to use cases, user stories, or features: Regardless of what requirement gathering techniques you use, map those requirements to a clearly defined set of tests.
7. Enable continuous integration: As you’ll learn in Chapter 9, continuous integration hinges on developers running the unit tests before the code is checked in. In addition, all unit tests should be executed in a build server at least once a day (hourly seems to work well as a build interval) or after any code has been checked in.
8. Don’t overtest: Test enough to cover functionality. Add new tests as defects appear. Use techniques such as equivalence partitioning and boundary value analysis to minimize the number of tests needed. Grouping, categorizing, and organizing your tests can ensure that you don’t write redundant tests and that the tests cover the range of features and scenarios for a given component.
-------------------------------------------------------------
No reading/improving knowledge outside the work / work hours
Inadequate productive time on the assigned tasks
- time to read/understand/seeking help/sitting with others to resolve the issues
- total in time
- incoming time
- breaks
- sense of urgency
Ownership
- role play
- assuming responsibility
- delivering stuff as committed
- we are missing releases due to bad quality
Not following the process
- Ex: Code migration, estimation, defect/INC status updates, Project kickstart meetings, code reviews
Not following the instructions
- Simple example is vacation calendar and Diwali vacation
- Production support start time
Interest or the lack of it
- to know what other people are doing and what we can learn from it
- to introduce new technical solutions
- to improve existing components
- to prevent repeat mistakes
- to know the whole system - people with long experience in the team are also restricting themselves to a smaller area
- we are STAGNANT as a team
Not having active conversations
- No heated arguments on how things should be done in terms of design / technology choice / schedule / quality
Leads:
Need to talk strong with other stakeholders
Need to have a clear plan on execution and proper tracking on it, communicate and record every change
Proper and sufficient documentation
Continuous knowledge improvement and sharing of knowledge
Process improvements
Productivity improvements
A default exception view - Any exception on any page must take the user to a custom error page if the exception is not handled.
The requests should be accepted only from whitelisted domains. If the referer is not available or if the referer is not in the whitelist, then an error message will be displayed.
---------------------------------------------
REST:
All REST URLs should be using HTTP POST from security perspective.
No input parameters must be present in the PATH variables - the URL/URI should be the generic representation of the resource. The important thing to consider for deciding the URI is the hierarchy of the resource and then the input parameters.
One URI must represent only one resource at the server side. One REST URL is pertaining to one URI. Do not use the same REST endpoint for diffent data.
For example, the postitions URI would be /holdings/positions/previousclose?account={account} rather than /holdings/positions/{account}/previousclose or /holdings/positions/previousclose/{account}. It can also be /holdings/positions?quotes=previousclose&account={account} if the same resource will handle both previous day and realtime dat.
In most cases, the resources respond with a collection. So use plurals in the URI. Eg: holdings/positions in the above URL.
Typically create two URLs (endpoints) per resource:
a. The resource collection (e.g. /holdings/positions - that returns a collection )
b. Individual resource within the collection (e.g. /holdings/positions/detail - that returns detail for one record )
Could respond with JSON/HTML/XML/Excel/PDF/Image content. The content type should be controlled by the HTTP Accept headers preferably or by parameters, not by different controller methods. Use content negotiators for producing different response types.
Use HTTP Error Codes in the response to communicate any system errors to the clients (Browsers) like resource not available, service down, etc and use the status/message attributes to present a meaningful message to the users.
Do not use nested objects in the response - the only nested(inner) object should be a list that contains the actual response. If you want to maintain OO structure / reuse the same classes, the use the mechanisms supported by the JSON libraries to avoid the nested objects in the response. For example, Jackson library offers an annotation called UNWRAP for flatening the inner objects.
Keep URLs as short as possible. Preferably, no more-than three nodes per URL.
Use nouns as resource names - do not use verbs. For example, process / getAccounts / getUsers / etc should not be used for READ transactions.
For any update opertions, use only POST. Do not user PUT/DELETE - thought REST spec suggests these, these should be disabled at the WebServer layer to mitigate any security risks.
Always send the charset code in the response.
Implement Access-Control-Allow-Origin config at the web server level.
Support a Maximum of 200 records per response. If possible, build support for Range/Content-Range HTTP headers to support better pagination.
Use the "Accept" header to support different versions of same resource.
For example, to retrieve a user in JSON format:
# Request
GET http://api.example.com/users/12345
Accept: application/json; version=1
# Response
HTTP/1.1 200 OK
Content-Type: application/json; version=1
{“id”:”12345”, “name”:”Joe DiMaggio”}
Now, to retrieve version 2 of that same resource in JSON format:
# Request
GET http :// api . example . com / users /12345
Accept: application/json; version=2
# Response
HTTP/1.1 200 OK
Content-Type: application/json; version=2
{“id”:”12345”, “firstName”:”Joe”, “lastName”:”DiMaggio”}
The requests should be accepted only from whitelisted domains. If the referer is not available or if the referer is not in the whitelist, then an error message will be displayed.
Use the below to disable caching and set these through a common utility function from the class like ResponseUtil.
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Jan 2015 00:00:00 GMT
E-Tag: hashofthetimestamp
Use Spring REST Docs to document the REST API.
Following are the different entities in the response.
•code – contains the HTTP response status code as an integer.
•status – contains the text: “success”, “fail”, or “error”. Where “fail” is for HTTP status response values from 500-599, “error” is for statuses 400-499, and “success” is for everything else (e.g. 1XX, 2XX and 3XX responses).
•message – only used for “fail” and “error” statuses to contain the error message. For internationalization (i18n) purposes, this could contain a message number or code, either alone or contained within delimiters.
•data – that contains the response body. In the case of “error” or “fail” statuses, this contains the cause, or exception name.
When Should I Create a New Version?
In API development there are many ways to break a contract and negatively impact your clients. If you
are uncertain of the consequences of your change it is better to play it safe and consider versioning.
There are several factors to consider when you are trying to decide if a new version is appropriate or if
a modification of an existing representation is sufficient and acceptable.
Changes that will break contracts
● Changing a property name (ie. “name” to “firstName”)
● Removal of property
● Changing property data type (numeric to string, boolean to bit/numeric, string to datetime, etc.)
● Validation rule change
● In Atom style links, modifying the “rel” value.
● A required resource is being introduced into an existing workflow
● Resource concept/intent change; the concept/intent or the meaning of the resource’s state has a different meaning from it’s original.
Examples:
○ A resource with the content type text/html once meant that the representation would be a collection of “links” to all supported media types, new text/html representation means “web browser form” for user input
○ An API populating an “endTime” on the resource “.../users/{id}/exams/{id}” once meant the student submitted the exam at that time, the new meaning is that it will be the scheduled end time of the exam.
● Adding new fields that came from an existing resource with the intent to deprecate the existing resource. Combining two resources into one and deprecating the two original resources.
○ There are two resources, “.../users/{id}/dropboxBaskets/{id}/messages/{id}” and “.../users/{id}/dropboxBaskets/{id}/messages/{id}/readStatus”. The new requirement is to put the properties from the readStatus resource into the individual message resource and deprecate the readStatus resource. This will cause the removal of a link to the
readStatus resource in the individual messages resource.
While this list is not full-inclusive, it gives you an idea of the types of changes that will cause havoc for your clients and require a new resource or a new version.
https://github.com/tfredrich/RestApiTutorial.com/raw/master/media/RESTful%20Best%20Practices-v1_2.pdf
---------------------------------------------
REST Utility
- read URL from a config (URLName:URL)
- Check for entitlements ??
- pack cookies (pick cookies to be packed from a config entry as diff apps may need diff cookies)
- timeout (configurable request timeout values per URL)
- Common error codes/messages with a facility to override from the consumer
- pagination
- scrollkeys should be passed to browser and then send back, the utility should handle packing/unpacking of scrollkeys
- have a standard name and pack/unpack autimatically based on the name / use an annotation
- logging
- audit
- error
- exception
- nppi masking
- security
- filter invalid characters of request
- perform output encoding of response
RestUtilConfig Sample:
URL=TARGET_URL:www.test.com|ROLE=000 - indicates do not check|CookiesToPack=abc.authservice,site|TO=30000(in milliseconds)|InputFilter=None|OutputEncode=Yes
Create a JS Utility that will help in packing the scrollkeys from browser in each request. Create it as a DOJO addon for the stores.
---------------------------------------------
Question:
Can we enforce a schema for JSON response?
---------------------------------------------
Questions:
How to set view names / URL aliases in Spring?
Spring MVC Training:
@Controller
@Service
@ModelAttribute
@RequestMapping
@ResponseBody
produces
ResponseEntity
HttpHeaders
Interceptors
AOP
Bean Validation:
bean{
@NotNull
@Max(5)
private Integer number;
@NotNull
@Future
@DateTimeFormat(iso=ISO.DATE)
private Date date;
}
@ExceptionHandler
@ControllerAdvice
@PostConstruct
@PreDestroy
Training:
DOJO
HTML 5
CSS 3
Responsive Web Design
Application Security
JVM Optimization
Java Performance Tuning
JUnit
Parameterized JUnit Testing:
http://examples.javacodegeeks.com/core-java/junit/junit-parameterized-test-example/
Selenium
JDBC
Dynatrace
BXP Operational activities
- ID Creation
- VM Creation
- Other VM related activities
- Release Baseline creation
- Code Deployment
- Create a wrapper script - do not have multiple steps/instructions - see if a UI can be created.
logback
Test driven development
-------------------------------------------------------------
Tips:
Spring Boot hosts an Initializr application at http://start.spring.io. The Initializr provides a Web
interface that allows you to enter project information, pick the capabilities needed for your project, and
voilà—it generates the project as a zip file.
-------------------------------------------------------------
A 2002 study by the Carnegie Mellon Software Engineering Institute (SEI) lists the following top 10 reasons why software projects fail:
• Inexperienced staff
• Lack of team cohesion and experience
• Lack of emphasis in using modern software-engineering practices
• Lack of a process or incorrect/insufficient emphasis in the application of a process
• Inadequate project management methodology
• Unclear, misunderstood, and undiscovered requirements
• Size (the larger the projects the more likely they are to fail)
• Lack of planning and estimating
• System-specific and technology-related issues being considered too late in the process
• New technologies and unforeseen problems
-------------------------------------------------------------
Not reading outside work
- If you dont read, you rely on others who may or may not have the expertise and most often you will end up with non-optimal solutions.
- If you read while executing projects, it might delay the execution, some time, you may want to redo the stuff, but will not be allowed as you will not have the time and budget.
-
-------------------------------------------------------------
Testing Best Practices
TDD advocates the practice of programming by writing tests first, writing the code to make the tests pass, then refactoring to optimize the code and the verall design of the application without breaking the tests. Obviously this leads to a comprehensive test suite that covers what your code does, because it was the contract of what was to be built.
1. Write tests first: Write tests that show the typical usage of a given class first (plus some boundary or edge cases). Then write enough code to pass the tests. Coding to interfaces as seen throughout the book can help make the test-first approach attainable.
2. Write code with clear intentions: Follow good object-oriented practices by creating code that makes its purpose obvious to others. Keep classes small and use good naming conventions. It is easier to write a test for a component for which the intent is known, especially since it is possible that somebody else might be writing the tests or enhancing them in the future. This practice becomes easier if you choose to use practice 1.
3. Test everything: If it can break, test it. If it can produce effects or output that’s not clear from its interface, test it. Even simple components can produce errors in other components if their intent or function is not clear. I equate tests in software to formal proofs in mathematics. Unless there is a comprehensive set of tests, any assurance of a component quality is just an illusion.
4. Establish a test harness early in the development: Set up testing as part of the build and if possible, use a test framework that integrates well with your development tools; doing this early in a project will ensure that the whole team embraces testing and that it becomes the path of least resistance.
5. Code enough to pass the unit tests: The task is complete when the unit test is complete. Just like code shouldn’t be checked into a version control system without compiling, code shouldn’t be checked in unless it has a comprehensive set of unit tests and those tests pass.
6. Map tests to use cases, user stories, or features: Regardless of what requirement gathering techniques you use, map those requirements to a clearly defined set of tests.
7. Enable continuous integration: As you’ll learn in Chapter 9, continuous integration hinges on developers running the unit tests before the code is checked in. In addition, all unit tests should be executed in a build server at least once a day (hourly seems to work well as a build interval) or after any code has been checked in.
8. Don’t overtest: Test enough to cover functionality. Add new tests as defects appear. Use techniques such as equivalence partitioning and boundary value analysis to minimize the number of tests needed. Grouping, categorizing, and organizing your tests can ensure that you don’t write redundant tests and that the tests cover the range of features and scenarios for a given component.
-------------------------------------------------------------
No reading/improving knowledge outside the work / work hours
Inadequate productive time on the assigned tasks
- time to read/understand/seeking help/sitting with others to resolve the issues
- total in time
- incoming time
- breaks
- sense of urgency
Ownership
- role play
- assuming responsibility
- delivering stuff as committed
- we are missing releases due to bad quality
Not following the process
- Ex: Code migration, estimation, defect/INC status updates, Project kickstart meetings, code reviews
Not following the instructions
- Simple example is vacation calendar and Diwali vacation
- Production support start time
Interest or the lack of it
- to know what other people are doing and what we can learn from it
- to introduce new technical solutions
- to improve existing components
- to prevent repeat mistakes
- to know the whole system - people with long experience in the team are also restricting themselves to a smaller area
- we are STAGNANT as a team
Not having active conversations
- No heated arguments on how things should be done in terms of design / technology choice / schedule / quality
Leads:
Need to talk strong with other stakeholders
Need to have a clear plan on execution and proper tracking on it, communicate and record every change
Proper and sufficient documentation
Continuous knowledge improvement and sharing of knowledge
Process improvements
Productivity improvements