Skip to content
This repository has been archived by the owner on Jun 4, 2019. It is now read-only.

Impossible to use utf #12

Open
pahanini opened this issue Jul 8, 2017 · 3 comments
Open

Impossible to use utf #12

pahanini opened this issue Jul 8, 2017 · 3 comments

Comments

@pahanini
Copy link

pahanini commented Jul 8, 2017

It seems something wrong with encoding or decoding of utf. Consider this test:

import (
	"github.com/smartystreets/mafsa"
	"github.com/stretchr/testify/require"
	"testing"
)

func TestRussian(t *testing.T) {
	a := mafsa.New()
	a.Insert("я") // I'm in russian
	a.Finish()
	a.Save("test")
	require.True(t, a.Contains("я")) // Fine

	b, err := mafsa.Load("test")
	require.NoError(t, err)
	require.True(t, b.Contains("я")) // Failure!!
}

func TestSpanish(t *testing.T) {
	a := mafsa.New()
	a.Insert("Gracías")
	a.Finish()
	a.Save("test")
	require.True(t, a.Contains("Gracías")) // Thank you in spanish

	b, err := mafsa.Load("test")
	require.NoError(t, err)
	require.True(t, b.Contains("Gracías")) // Failure!!
}
@mdwhatcott
Copy link
Contributor

@mholt - Does mafsa support non-ascii characters?

@pahanini
Copy link
Author

pahanini commented Jul 9, 2017

@mholt Not sure but I did find any alphabet limitations here. It works with non-ascii but it seems an error during load/save process. Look at the test it works fine with buildTree and fails only after I load it from file and start to use minTree

@pahanini
Copy link
Author

pahanini commented Jul 9, 2017

It seems merging this PR will solve this issue #8

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants