RealJon

Predicate fields are indeed an oddity, but not an architectural one - it's for situations where the documents need to specify criteria (predicates) for when they should match - like only match for certain users, certain times of day etc. It's probably an underused feature imho since most people don't know this can be done efficiently.

If you have dynamic fields like in your SaaS example I recommend using a single map field rather than let data not under your control drive changes to the set of fields.

> Do you know how long a package update takes in Vespa, to add, say, a single field?

A few seconds. However, rather than having operators do any of this manually, set up an automatic process which deploys on each change made to the repo (i.e do CD).

> all the reference documentation in one place

https://docs.vespa.ai/documentation/api.html

RealJon

9 年前discuss

Write speed (add or update) is typically between a few thousand to a few tens of thousands operations per second per node sustained, depending on sizeof data etc.

Sustaining throughput over long time is important and often overlooked mentioned in benchmarks.

RealJon

9 年前discuss

No.

RealJon

9 年前discuss

You don't need to plug in any Java code, you can use it with HTTP calls to read and write.

No plans to break it up into pieces (apart from already consisting of modules). It does one thing, it just happens to be a big thing :-)

If you have a mac of Linux box you can have it up and running in 10 minutes. Multi-node production deployments are no different because Vespa manages the nodes, not you directly.

RealJon

9 年前discuss

No, this is run by a single, very cohesive, very remote team.

There is some misidentification in your list (our .def files are nothing to do with Teamcenter). We use 2 languages because we Java and C++ have different strengths which make each suitable at different layers of the architecture. The rest is a combination of "for good reasons" and "leftover scraps" :-)

RealJon

9 年前discuss

TensorFlow on Spark is for learning, Vespa is for serving. Where Vespa excels is in evaluating a learned model very quickly over lots of documents. We're working on providing support for running models learned with TensorFlow directly. For now people make the translation on their own.

RealJon

9 年前discuss

Not precisely sure where we were in 2011, but I think these are the biggest ones that came after, off the top of my head (i.e sure to be missing something):

  - Merging content and index clusters to one to make index clusters elastic and auto-recovering on data loss.
  - Fully realtime writes.
  - Support more advanced machine-learned ranking through tensors.
  - Streaming (personal) search supporting a large write rate.
  - Document references.
  - WAND and RANK operators.
  - Rank features over multivalue text fields.
  - Predicate fields.
  - Lots and lots of performance work.

RealJon

9 年前discuss

Sorry about that - we haven't really optimized the module structure for newcomer comprehension. If you tell me what you want to are looking for I can point you to the right place.

Submissions

[untitled]

Don't use ANN for vector search in personal data

ANN support announced in Vespa.ai: Vector and filters and text search at scale

On choosing an (ANN) algorithm from the literature for production

Why most computation will become online

The big data maturity levels

An open source scalable shopping back end w. navigation, search, recommendation

Serving article comments using reinforcement learning of a neural net

Implementing the next Gmail efficiently in open source

Scaling TensorFlow inference to unlimited items per request with bounded latency

Blog recommendation: Evaluating neural nets at serving time using open source

Scalable evaluation of ONNX models with open source Vespa

Real time distributed joins: Possible? Some times

Benchmark: Scaling TensorFlow model evaluation using Vespa

How we made neural nets inference 20x faster with Vespa

Scalable realtime blog recommendation using neural nets with Vespa

comments