Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

QuestionOutcome
  1. How do we accommodate different data schema that allow for multiple data stream?
 See requirement #1

2. Does Distill require a specific backend (Elastic) or can it go to Solr/Lucene

Underlying data store needs to support key value pairs
3. How do we support Windows Users?
  • Investigate whether we are using packages that don't build in Windows
  • Integrate testing across platforms

    See requirement #9

    4. How do we provide the "average" data scientist enough out of the box packages, modules to be minimally viable out of the box? 
    5. Roadmap for supporting packages and Anaconda distribution 
    6. Migrate to Django from Flask? 
    7. Is Distill simple python, or does it run as a service (or on a webservice) by design? 
    8. Does Distill manage scale in its connections to other datastores, or does it rely soley on Lucene based services (Elastic)?Distill's querying is dependent on how well Elasticsearch scales on query.
    9. Does Distill remain tethered outright to Elastic? See requirement #2
    10. TLS or SSL: Modern vs. Legacy network support. 

    Analytics & Processing Examples

    These examples are here for drawing out higher-level goals for Distill's functionality. This section can be removed once the goals have been solidified.

    • Build intervals from matching sequences of raw events
    • Filter out unwanted events
      • Noisy/irrelevant events
        • May be conditional on neighboring events
      • "dangling" events (e.g. a stop event with no corresponding start)
    • Collapse duplicate events into a single event (when is this preferable to creating an interval?)
    • Create "sandwiches" (a set of events bookended by, e.g., a related start and stop event)
    • Replace some logs/data with other logs/data

    ...