导入sogou输入法的细胞词库

Posted on July 16, 2010 by yongsun

首先请下载这个导入工具，解压缩到某个目录中。从pinyin.sogou.com下载细胞词库（scel格式），保存在某个目录中。然后在terminal中执行下面的操作：

$ python import_sogou_celldict.py PATH_TO_CELL_DICT.scel

感谢中文社区其他开发者和项目reverse engineering的工作，

98 thoughts on “导入sogou输入法的细胞词库”

fen on July 16, 2010 at 8:55 am said:

好东西，这回用的顺手多了
Ryan on July 16, 2010 at 9:56 am said:

Long expected!! Thanks!!!
圈叔 on July 16, 2010 at 11:13 am said:

"cellldict.py"多了个l 应该是"celldict.py"
圈叔 on July 16, 2010 at 11:16 am said:

great! 效果很赞!!!
Leeiio on July 16, 2010 at 12:04 pm said:

光导入我觉得不够，得能自动更新才好。
catholicon on July 16, 2010 at 1:29 pm said:

一个不懂命令行的人求详细步骤⋯⋯
arne on July 16, 2010 at 1:35 pm said:

导入计算机细胞词库，http://pinyin.sogou.com/dict/cell.php?id=15117
代码出错，
Please specify the Sogou PinYin Cell dict file!
arnes-workstation:sunpinyin_importer arne$ ./import_sogou_celldict.py ~/Downloads/download_cell.php
Traceback (most recent call last):
File "./import_sogou_celldict.py", line 81, in
main()
File "./import_sogou_celldict.py", line 78, in main
import_to_sunpinyin_user_dict (generator)
File "/Users/arne/Downloads/sunpinyin_importer/importer.py", line 52, in import_to_sunpinyin_user_dict
print "[%s] is already in sunpinyin's sysdict" % utf8str
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-4: ordinal not in range(128)
yongsun on July 16, 2010 at 1:53 pm said:

@圈叔，多谢指正！已修正
yongsun on July 16, 2010 at 1:53 pm said:

@arne，感谢您的反馈，我晚上回去调试一下 ...
yongsun on July 16, 2010 at 1:55 pm said:

@catholicon，在Applications/Utilities下，打开terminal，然后运行文中的命令
yongsun on July 16, 2010 at 1:56 pm said:

@Leeiio，我们人力有限，还有其他许多重要的feature要实现，恐怕自动更新词库要等比较长的时间了 ... 非常抱歉 ...
cbkid on July 16, 2010 at 3:15 pm said:

greader过来帮顶，太NB了。太感谢了
圈叔 on July 16, 2010 at 6:37 pm said:

如何导出自己的sunpinyin 词库呢?
Bill on July 16, 2010 at 7:22 pm said:

原来那个 importer.py 每转换一个词条就commit一次，导入速度比较慢，我自己修改了一下，每100个词条commit一下，导入速度快多了。
--- sunpinyin_importer/importer.py 2010-07-15 00:01:15.000000000 +0800
+++ sunpinyin_importer/importer.py 2010-07-16 19:05:31.615595203 +0800
@@ -36,7 +36,9 @@
utf8str TEXT, UNIQUE (utf8str));
"""
db.executescript (sqlstring)
-
+
+ count = 0
+
for (pystr, utf8str) in records:
try:
syllables = [valid_syllables[s] for s in pystr.split("'")]
@@ -71,12 +73,14 @@
"""
try:
db.execute (sqlstring, record)
- db.commit ()
+ count += 1
+ if count % 100 == 0:
+ db.commit ()
print "[%s] is imported into sunpinyin's userdict" % utf8str
except:
#print "[%s] is already in sunpinyin's userdict" % utf8str
pass
-
+ db.commit()
db.close()
yongsun on July 16, 2010 at 8:31 pm said:

@Bill, 非常感谢您的patch，已更新
yongsun on July 16, 2010 at 8:34 pm said:

@arne，不知道您是否使用的是power的系统，或者下载的文件有误。我刚刚尝试导入了这个文件，没有碰到这个问题 ... 如果是power系统，您可以尝试重新下载importer包，再导入一次试一试，刚刚修改了一些可能是iconv目标编码的问题 ...
yongsun on July 17, 2010 at 1:02 am said:

@圈叔，麻烦你重新下载importer.tar.bz2，解开之后，执行python importer.py > userdict.txt，就可以了 ...
arne on July 17, 2010 at 6:06 am said:

@yongsun,回家测试了一下，好像没问题，
可能是公司里限制上网，必须使用代理服务器的问题。
谢谢
你们 on July 17, 2010 at 9:06 am said:

导入唐诗经常会出现：[霓裳一曲千峰上] is too long or too short for sunpinyin userdict
can. on July 17, 2010 at 10:48 am said:

看到这个feature,觉得终于可以替代fit了...
one more thing: 中英文切换的时候,怎么只有换到英文的时候有growl的提示呢,总觉得很不踏实...还有,怎么删除词库里错误的词条啊?
yongsun on July 17, 2010 at 4:24 pm said:

@你们，sunpinyin的用户词典有字数限制，比较长的用户输入是通过history cache来做补充的。
yongsun on July 17, 2010 at 4:26 pm said:

@can，ctrl+command+num是删除用户自造词的快捷键 ... growl的提示其实不是很理想，我们今后会考虑用自己的OSD实现 ...
MARCZAI on July 20, 2010 at 10:41 pm said:

我最近看了加加输入法的好多词库都是txt的，假如可以导入的话，岂不是可以更好的弥补词库的不足。加加词库的格式是：
企业
这zhe个
以后
这zhe些
应该
进行xing
政府
yongsun on July 21, 2010 at 12:04 am said:

@MARCZAI，不知道加加输入法的词库license是怎样的 ...
Lya on July 21, 2010 at 12:09 am said:

我下下来的搜狗细胞字库是乱码，然后硬在终端里运行，被告知，permission denied。。。这要怎么解决啊。。。
oldbear on July 21, 2010 at 12:22 am said:

感谢开发出这么好用的输入法！
问一下啊，有没i有大侠顺手做一个可以导入谷歌拼音输入法的词库的？
可能很多人像我一样，已经用谷歌积累了很长时间的私人词库，这比搜狗的细胞词库要贴心得多。谷歌拼音输入法支持导出词库，txt格式，这应该比细胞词库转换到sunpinyin更简单吧？
求大侠出手！
yongsun on July 21, 2010 at 7:16 am said:

@Lay，您是说文件名是乱码？改个英文的名字试试看 ...
yongsun on July 21, 2010 at 7:18 am said:

@oldbear，我们已经提供了多种输入法用户词库的导入脚本，其中也包括google拼音的，参见http://yongsun.me/tag/importer ...
MARCZAI on July 21, 2010 at 8:25 am said:

加加输入法的词库license,因为很多txt的词库都是网友制作的，他们很乐意共享的。FIT的大词库版本就是我在加加论坛上让beckyer制作的。
论坛地址：bbs.jjol.cn
yongsun on July 21, 2010 at 8:49 am said:

@MARCZAI，非常感谢您提供的信息，我会找时间看看如何实现一个导入脚本 ...
圈叔 on July 24, 2010 at 2:45 pm said:

请求加入功能选择输入繁体还是选择输入简体
偶觉得很多时候都有这个需求的！
yongsun on July 25, 2010 at 9:07 am said:

@圈叔，在计划中，借助opencc来实现 ...
翻墙砖家 on July 27, 2010 at 11:14 pm said:

我过来只是为了对你表示感谢，呵呵
非常好的输入法，非常棒！
我代表所有（愿意让我代表的） mac 用户感谢你
pensz on July 31, 2010 at 12:21 am said:

非常感谢作者提供这么好的程序。
pulp on August 2, 2010 at 3:10 am said:

请问如何调整词频？比如我输入「d」，出来的第一个字是「到」，我想调整成「的」，如何实现啊？
yongsun on August 2, 2010 at 8:22 am said:

@pulp，目前sunpinyin的单字词的词频调整还有些问题，我们会尽快改进 ...
Lex on August 4, 2010 at 1:19 am said:

Permission denied,何解啊~~~求助！！T_T！！
yongsun on August 4, 2010 at 10:41 am said:

@Lex，估计是您直接执行.scel文件了？
圈叔 on August 8, 2010 at 1:23 pm said:

偶发现一个奇怪的问题,偶输入liangnv 可以打出靓女
但是偶输入liang 却找不到liang 偶选择的字符集是GB18030
yongsun on August 9, 2010 at 7:48 am said:

@圈叔已在open-gram github上fix了，会在2.0.2.1008正是版本中集成进来…
boboweb on August 10, 2010 at 9:06 am said:

导入的词库过多，输入的时候就特别的卡
有明显迟钝的现象
fred on August 12, 2010 at 4:02 pm said:

为什么我导入的时候出现的是 XXXX is too long or too short for sunpinyin userdict?
dunjian on August 17, 2010 at 9:44 am said:

导入时出现以下错误，还请指教：
Traceback (most recent call last):
File "import_sogou_celldict.py", line 81, in
main()
File "import_sogou_celldict.py", line 78, in main
import_to_sunpinyin_user_dict (generator)
File "/Users/Xinfeng/Desktop/sunpinyin_importer/importer.py", line 50, in import_to_sunpinyin_user_dict
print "[%s] is too long or too short for sunpinyin userdict" % utf8str
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-9: ordinal not in range(128)
yongsun on August 21, 2010 at 11:23 am said:

@dunjian，您能告诉我导入的是那个细胞词库么？我有时间尝试一下 ...
dunjian on August 22, 2010 at 2:03 am said:

谢谢，我下载的是搜狗精选词库，链接为： http://pinyin.sogou.com/dict/cell.php?id=11826 另外，在导入的时候，我把文件名改为1.scel了，不知道有没有影响。
Houge on August 22, 2010 at 10:00 am said:

我导入 http://pinyin.sogou.com/dict/cell.php?id=15125的词库时，出现下面的错误，请指教
su-yeqindeibook-g4:sunpinyin_importer suyeqin$ python import_sogou_celldict.py /Users/suyeqin/yixue.scel
Traceback (most recent call last):
File "import_sogou_celldict.py", line 81, in
main()
File "import_sogou_celldict.py", line 78, in main
import_to_sunpinyin_user_dict (generator)
File "/Users/suyeqin/sunpinyin_importer/importer.py", line 58, in import_to_sunpinyin_user_dict
sysdict = load_system_dict()
File "/Users/suyeqin/sunpinyin_importer/importer.py", line 47, in load_system_dict
for w in str.decode('UTF-32').split(''):
LookupError: unknown encoding: UTF-32
Houge on August 22, 2010 at 10:14 am said:

我在导入我的Google用户词库的时候，一样出现了一下错误，请作者看一下
我的笔记本是iBook G4，操作系统是Mac OS X 10.5.8
su-yeqindeibook-g4:sunpinyin_importer suyeqin$ python import_google_userdict.py /Users/suyeqin/Google.dic
Traceback (most recent call last):
File "import_google_userdict.py", line 33, in
main()
File "import_google_userdict.py", line 30, in main
import_to_sunpinyin_user_dict (google_user_dict)
File "/Users/suyeqin/sunpinyin_importer/importer.py", line 58, in import_to_sunpinyin_user_dict
sysdict = load_system_dict()
File "/Users/suyeqin/sunpinyin_importer/importer.py", line 47, in load_system_dict
for w in str.decode('UTF-32').split(''):
LookupError: unknown encoding: UTF-32
su-yeqindeibook-g4:sunpinyin_importer suyeqin$
o_0! on August 22, 2010 at 11:53 am said:

还真是一点不懂 http://pinyin.sogou.com/dict/cell.php?id=11826 用这个的要怎么导入阿？？
yongsun on August 22, 2010 at 12:41 pm said:

@Houge，貌似10.5上的python不支持UTF-32的encoding？试一试UCS4？或者实在不行就在其他系统上导入后，再copy过来
yongsun on August 22, 2010 at 12:47 pm said:

@dunjian，我刚尝试导入了这个精选词库，貌似没有问题，能否重新下载最新的importer脚本再试一次？

素心如何天上月

Yong Sun's Blog

导入sogou输入法的细胞词库

98 thoughts on “导入sogou输入法的细胞词库”

Leave a Reply Cancel reply