Pipeline, execution and schedule views performance issues fix #547
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pipeline, execution and schedule views were implemented very uneffective. When creating view (table), hundreds of selects were unnecessary executed in database for such trivial actions as check of permission for buttons, etc.
This is unacceptable in production environment where database is on separate server and thus delays occur.
With such a high count of selects (120 selects for table with 20 entries per page per refresh) we experienced heavy performance issues - refresh of table ~10 seconds.
Simple solution - caches were introduced to store objects which are so often queried.
This way, no redundant selects are executed, performance has gone up rapidly.
However, this is still not optimal solution as these caches are per user (as already caches introduced before) and are influenced by objects that user cannot even see.
There is concept in all these views, that refresher checks if any pipelines / executions have been changed in some time. However, there is no regard to access permissions. If any pipeline / execution has been changed, even if user cannot see it, table and caches are reloaded. In fact only changes of objects that user can access should trigger refresh.
Moreover, cache should be static and general for all users and not per user. It should be loaded at start and then dynamically changed when objects change and then in some larger intervals - not every time somtehing changes in the database.
However, the described problems are much more complicated as there is no architecture to support the right way of doing this.
Introduced caches solve the most pressing performance issues very effectively.
Resolves #546
partly resolves #544