Date
Meeting link: https://meet.google.com/uny-zfca-sny
Time: 9 am pacific time
Links: TimeAndDate.com
Attendees
- Andrey Ukhanov (and other Bloomberg folks)
- Christopher Ball
- David Smiley
- Jan Hoydahl
- Jason Gerlowski
- Marcus Eagan
- Michael Gibney
- Mikhail Khludnev
- (sorry if I missed anyone - please feel free to add others in attendance!)
Discussion items
| Time | Item | Who | Notes |
|---|---|---|---|
| RateLimiter fixes/features | |||
| Solr "Vision" Discussions | Christopher Ball | ||
| Conference Summary and Follow-Ups | David Smiley | ||
| Recent Lucene w/ GPU Efforts | Marcus Eagan |
Summary
Rate Limiting
Michael Gibney started us off with some good discussion around some proposed changes to Solr's rate-limiter functionality.
His main aim was to raise awareness of discussions going on in PR 2522. Theoretically, rate-limiting currently supports a feature called "slot borrowing", where a request up against the limit for its request-type may "borrow" an execution-slot from another request-type that's under its own limit. But in practice this feature has no user impact, as rate-limiting currently only supports a single request-type: queries. The PR wants to make some improvements and changes to how "slot borrowing" works, but was/is wrestling with how strictly to observe backcompat in this scenario. Various attendees promised to chime in on the PR discussion to help drive consensus there.
From there the discussion meandered to some other aspects of Solr's rate-limiting implementation: the inability to distinguish between internal and external requests, comparisons to the similar but distinct Circuit Breaker feature-set, and some gaps in Solr's metric-gathering that might represent overall load more accurately in some cases.
Solr "Vision" Sync Ups
Inspired in part by the sort of high-level discussions that this month's conferences (Berlin Buzzwords, Community over Code EU) helped facilitate, Christopher Ball wondered aloud whether Solr might benefit from having regular high-level discussion around the direction and vision for the project. Providing a periodic opportunity for developers to zoom out, share larger initiatives (either planned or completed), and maybe discuss industry developments might aid in release planning, help developers find opportunity for collaboration with others, etc.
Reception to the idea seemed generally positive, though there was some uncertainty what this would look like or how it might be facilitated in practice. The most concrete proposal was a Virtual Meeting held 2-4 times a year for discussing these sort of higher-level "Vision" topics. Jan and others suggested that the meeting might start by going over the current state of either the Solr Roadmap, or SIP List, or both.
The next step seems to be to refine the idea a bit, and for someone to take the initiative to write it up and propose it on the dev-list.
Conference Summary and Follow-Ups
David shared with us some discussions he'd had with Uwe & Varun Thacker on some points that Solr should improve upon. In particular, Michael and David considered Solr's readiness to start making use of Lucene's per-segment filter caches as a replacement or supplement to the existing whole-index filter cache that Solr currently uses. Secondly, the use of a single Jetty ThreadPool at 10k limit is problematic; should either be split up for have inner ThreadPools for requests of different types. Both these are worthy of Solr "roadmap" topics.
Lucene w/ GPU Efforts
[Jason: I dropped from the call right as this was starting and can't speak to the discussion. Marcus Alexander Eagan or anyone else in attendance - could you fill this in if you get a chance and are willing to share? ]