Software Engineering for Smart Data Analytics & Smart Data Analytics for Software Engineering

User Tools

Site Tools

Query Caching and Scheduling

This page describes current problems with the recomputing and caching in Cultivate and propose a possible solution that makes caching, query scheduling and notification explicit. The solution is made of the components Repository (as cache) and Refresher (as scheduler).

Problem: Issuing a Prolog query while the project is in building mode is currently possible, but returns wrong results.

→ Rework to the notion of deferred query execution. Calling query.execute() should tell the prologQueryExecutor to schedule this query. he PrologQueryExecutor should queue query requests and only execute them, when factbase has been constructed.

Problem: Notification is not explicit and scattered. Currently there is a CultivateInvalidationBuilder, UpdateJob and an DiagramViewUpdateJob.

→ Rework Builder (and UpdateJob? or delete this) to do the work, recompute all cached queries (smells, metrics, other queries) and then notify all interested clients. All Views (DiagramViewPart, ResultViewPart, CloudViewPart) should register for certain notification-topics (and content) at a central place - Repository in cultivate.core (sounds like an Event Bus with Pub/Sub).

Problem: No manual refreshing of cultivate caches is possible

→ Add an action into the ui to request a complete clear and rebuild of the cultivate cache. The automatic refresh sometimes does not work, when the factbase was recreated.

Problem: Rebuilding the cache is called three times for each change of the JT factbase

→ With this described reworking, we should also explicitly have a look at this and call an cache rebuild only once.

Possible Problem: Cultivate starts only upon first access to CultivateNatureUtil.getCultivateNature()

Further, Cultivate restarts the factbase upon switching between projects. Is this because of JT or is the factbase still valid in memory, so no rebuild is needed?

dsp: What does 'restarting' the factbase mean?

More Notes on this

Repository and Scheduler run outside the UI thread, notifications are the border into the ui thread - clients have to take care of reading and updating their views in ui thread. All computation should go in the other thread, issued by scheduler.

This way, asynchrony is made explicit within the event bus inside the repository. Borders to other threads are also explicit, as all of them may only be located inside the handlers for notification events (part of UI controllers in the future). Transition to ui thread is explicit there.


The solution depends upon further reworking / splitting detector into the concept of a query (is already there) with sub-concepts: smelldetector, metric, other special queries

Possible problems

Writing a simple event bus with pub/sub is easy. Adding correct threading into it and writing the queuing for the scheduler is harder.

dsp's remarks

  • One design goal of the current implementation was to re-execute queries only when necessary, so that no query is executed in case the developer works with the Java perspective and has no Cultivate view open at the moment.
  • AFAIUnderstand your suggestions are a more precise description of the restaurant metaphor, so that I aggree with the general direction. (Scheduling the architecture changes is of course up to the planning game.)
teaching/labs/xp/2009a/query_caching_and_scheduling.txt · Last modified: 2018/05/09 01:59 (external edit)

SEWiki, © 2021