chore(embeddings): use framework embeddings, refactor ai providers

[jezekra1]

BREAKING CHANGE:

unification of LLM_BACKEND, EMBEDDING_BACKEND -> AI_BACKEND

open for discussion

[jezekra1]

maxSerializedLength is not affecting the markdown output.

I removed the restrictions on max content length from markdown as well as the simplified extraction:

extraction: { fields: { markdown: {} } },

Previously we had table extraction disabled and the output was truncated to 25k characters due to slow embeddings, however the issue has mostly been resolved and we can include this data again.

Also this makes our implementation more aligned with framework defaults.

[Tomas2D]

The maxSerializedLength property affects only the serialized output, which contains the markdown output and it works correctly.

    const instance = new WikipediaTool({
      output: {
        maxSerializedLength: 100,
      },
    });
    const response = await instance.run({
      query: "ice hockey",
    });
    expect(response.getTextContent()).toHaveLength(100);

Source: https://github.com/i-am-bee/bee-agent-framework/blob/6332b0c3a6cb82310fdcef7f576c4a5a6e2d40fd/src/tools/search/wikipedia.ts#L100-L111

[jezekra1]

But getTextContent aggregates text from all documents, which is not what we want here, because we add aditional information about the source to each chunk

[Tomas2D]

Just a few suggestions. Good work 👍🏻

[Tomas2D]

The maxSerializedLength property affects only the serialized output, which contains the markdown output and it works correctly.

    const instance = new WikipediaTool({
      output: {
        maxSerializedLength: 100,
      },
    });
    const response = await instance.run({
      query: "ice hockey",
    });
    expect(response.getTextContent()).toHaveLength(100);

Source: https://github.com/i-am-bee/bee-agent-framework/blob/6332b0c3a6cb82310fdcef7f576c4a5a6e2d40fd/src/tools/search/wikipedia.ts#L100-L111

[pilartomas]

This is weird, why isn't markdown a string already? If it can be something else, we must handle it or fail.

[pilartomas]

Also, is markdown good input type for splitString? 🤔 Splitting tags might cause problems.

[jezekra1]

It is typed as unknown, but we already have this in the examples:

https://github.com/i-am-bee/bee-agent-framework/blob/33f1db5971d269935878a4a83a484a44371c3cdd/examples/tools/custom/piping.ts#L29

[pilartomas]

That might be just for simplicity. We need to make sure it is type-safe here. With a typeof else throw guard if necessary.

[pilartomas]

Please make sure the markdown handling is safe at runtime, otherwise LGTM 👍

[jezekra1]

Please make sure the markdown handling is safe at runtime, otherwise LGTM 👍

Actually, it could be null (src), the document will now be skipped in such case.

[pilartomas]

It is still types as unknown though, the typecast is just not safe there.

[jezekra1]

It is still types as unknown though, the typecast is just not safe there.

@Tomas2D is this fix correct?

i-am-bee/bee-agent-framework#267