请选择 进入手机版 | 继续访问电脑版

『瀚思彼岸』» 智能家居技术论坛

 找回密码
 立即注册
查看: 469|回复: 4

[插件发布] 实时获取小爱音响解析返回结果

[复制链接]

1

主题

3

帖子

65

积分

论坛分享达人

积分
65
金钱
62
HASS币
0
发表于 2020-6-22 22:07:12 | 显示全部楼层 |阅读模式
本帖最后由 zimiss 于 2020-6-28 22:36 编辑

经过最近的不懈努力,终于小有结果,现在删除之前的分析过程(主要是没人看)。实现原理是通过向/usr/bin/mipns-xiaomi进程注入代码,劫持mibrain_asr_nlp_record_write函数的调用。
现在没有实现复杂的功能,只是把收到的结果直接输出,还没有写收到之后的处理,所以还不能使用,不过可以测试一下。

如何测试
直接kill掉/usr/bin/mipns-xiaomi这个进程,自己手动在shell中启动,然后执行[url=]get_reponse,之后回车退出。[/url] [url=]libmy.so这个文件和get_reponse放到一个目录中(可能是/data/这个目录)。[/url]
再使用小爱的时候就会在控制台看到输出结果了。

存在问题
1 动态库中的函数还无法正确调用正常的mibrain_asr_nlp_record_write,导致我测试的时候程序直接崩溃了一次,暂时还不知道具体原因。
2 注入的时候使用的都是绝对地址,版本不一致可能无法使用,我使用的版本是1.58.15

get_reponse运行结果

[email protected]:/data/get# ./get_response 
targetpid is :2937
addr = -152924160
remote_func_addr:0xf6e29000
mapbase is 0xf6ef4ebc
[+] Calling mmap in target process.
func_addrs 0xf6ef4ebc
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from mmap, return value=f77e0000, pc=0 
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
[+] Get imports: dlopen: f7123a88, dlsym: f7123b74, dlclose: f7123b14, dlerror: f7123e1c
library path = /data/libmy.so
[+] Calling dlopen in target process.
func_addrs 0xf7123a88
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from dlopen, return value=397d40, pc=0 
[+] Calling dlsym in target process.
func_addrs 0xf7123b74
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from dlsym, return value=e531ae45, pc=0 
hook_entry_addr = 0x397d40
[+] Calling my in target process.
func_addrs 0xe531ae45
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from my, return value=1, pc=0 
addr = -149356544
remote_func_addr:0xf7190000
mytesthook_addr = 0xf71bb1a8
Press enter to dlclose and detach
/usr/bin/mipns-xiaomi运行结果
[email protected]:~# /usr/bin/mipns-xiaomi -c /usr/share/xiaomi/xaudio_engine.conf -r opus32 -l
this is Lx06  
Hardware PCM card 0 'AML-AXGSOUND' device 3 subdevice 0
Its setup is:
  stream       : CAPTURE
  access       : RW_INTERLEAVED
  format       : S16_LE
  subformat    : STD
  channels     : 8
  rate         : 48000
  exact rate   : 48000 (48000/1)
  msbits       : 16
  buffer_size  : 8192
  period_size  : 512
  period_time  : 10666
  tstamp_mode  : NONE
  tstamp_type  : MONOTONIC
  period_step  : 1
  avail_min    : 512
  period_event : 0
  start_threshold  : 1
  stop_threshold   : 8192
  silence_threshold: 0
  silence_size : 0
  boundary     : 1073741824
  appl_ptr     : 0
  hw_ptr       : 0
this is Lx06 mic detect methods 
this is Lx06 position in paramerers.c 
this is Lx06 position in mvdr.c  
/usr/share/xiaomi/dnn_vad_model/model_dense_1_0_transpose.txt 
dense layer1 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_1_1_transpose.txt 
dense layer1 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_2_0_transpose.txt 
dense layer2 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_2_1_transpose.txt 
dense layer2 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_3_0_transpose.txt 
dense layer3 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_3_1_transpose.txt 
dense layer3 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_4_0_transpose.txt 
dense layer4 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_4_1_transpose.txt 
dense layer4 bias initialized !
all the model initialized !
the DNN model size is 0 
finish dnn inital
xaudio: dnn wrapper init path /usr/share/xiaomi/dnn_we_model
vad model version is: vad_2.1.0_0
min_voice_length=80, min_sil_length=50
/data/libmy.so
/data/libmy.so
rice wakeup 1-level: frame=24005, start frame=23937, end frame=24005, at 2020-6-28-22-29-24.703729
rice wakeup 2-level: frame=24005, start frame=23939, end frame=24005, at 2020-6-28-22-29-24.839266
{
        "code": 0
}
idle
json_object_from_file: error opening file /data/upnp-disc/devices.json: No such file or directory
{"meta":{"type":"RESULT_NLP","nlp":{},"request_id":"f8c9cba402b6e765caa76bf17d4c6f5a","timestamp":1593354566243},"response":{"status":{"code":200,"error_type":"success"},"answer":[{"domain":"soundboxControl","action":"next","text":"","content":{"to_speak":""},"intention":{"score":0.95,"query":"下一曲","domain":"soundboxControl","action":"next","originAction":"next","func":"play_control.switch.common.next","complete":true,"ignoreSpeak":false,"intent_arbitrator_info":{"predict_result_domain":"soundboxControl","func":"play_control.switch.common.next","l2_domain":"soundboxControl","l2_func":"play_control.switch.common.next","score_domains":{"fuzzySearch":{"score":0.1,"func":"knowledge.fuzzy_search.common.common"},"default":{"score":0.05,"func":""},"music":{"score":0.4,"func":"play_control.switch.common.next"},"soundboxControl":{"score":0.95,"func":"play_control.switch.common.next"}},"arbitrator_level":"","dialog_status":"FINISH","single_result_domain":"soundboxControl","multi_result_domain":"soundboxControl","prejudge_status":"PREJUDGE_NOT_AFFECTED"},"domain_judge":"judge","rewrite_infos":{}}}],"version":"2.2","session_id":"8630dc0ab4a147cfa974600c03ba2726","request_id":"f8c9cba402b6e765caa76bf17d4c6f5a","latency":249,"eid":"0:0:0:0:0:0:0:101:0:0:0:0:0:34:0","instructions":[{"header":{"namespace":"Execution","name":"InstructionControl","id":"c9a0274df05a4c619f1c90dc667425a1","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{"behavior":"INSERT_FRONT"}},{"header":{"namespace":"PlaybackController","name":"Next","id":"93458338ff4e450d83aab6323d63320b","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{}},{"header":{"namespace":"Suggestion","name":"ShowContextSuggestions","id":"7d6c7980440c420498e258bf5cb7fb81","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{"cloud_control":{"show_layout":2,"with_double_quotation":false},"suggestions":[{"query":"开启位置信息","send_query":"开启位置信息","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]},{"query":"进入飞行模式","send_query":"进入飞行模式","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]},{"query":"打开震动","send_query":"打开震动","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]}]}}],"tts_enable":false}}
{
        "code": 0
}
idle

主要根据github上的一个代码进行适配修改的,在这个系统里面暂时找不到那个链接了,之后再放上了吧。

libmy.so

16.36 KB, 下载次数: 0

get_reponse

20.91 KB, 下载次数: 1

评分

参与人数 1金钱 +20 收起 理由
+ 20 感谢楼主分享!

查看全部评分

回复

使用道具 举报

2

主题

90

帖子

450

积分

中级会员

Rank: 3Rank: 3

积分
450
金钱
360
HASS币
0
发表于 2020-6-23 15:45:15 | 显示全部楼层
{:3_41:具体要怎样用呢?
回复

使用道具 举报

8

主题

367

帖子

1101

积分

金牌会员

Rank: 6Rank: 6

积分
1101
金钱
734
HASS币
0
发表于 2020-6-28 19:13:01 | 显示全部楼层
刚刚 快要成效了。。。然后上班了
回复

使用道具 举报

1

主题

3

帖子

65

积分

论坛分享达人

积分
65
金钱
62
HASS币
0
 楼主| 发表于 2020-6-28 22:54:09 来自手机 | 显示全部楼层
yylwhy 发表于 2020-6-28 19:13
刚刚 快要成效了。。。然后上班了

半成品算是完成了٩( •̀㉨•́ )و get!
回复

使用道具 举报

0

主题

35

帖子

114

积分

注册会员

Rank: 2

积分
114
金钱
79
HASS币
0
发表于 6 天前 | 显示全部楼层
感谢楼主分享~
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

Archiver|手机版|小黑屋|Hassbian

GMT+8, 2020-7-8 08:03 , Processed in 0.134003 second(s), 31 queries .

Powered by Discuz! X3.4

© 2001-2017 Comsenz Inc.

快速回复 返回顶部 返回列表