实时获取小爱音响解析返回结果

zimiss

经过最近的不懈努力，终于小有结果，现在删除之前的分析过程(主要是没人看)。实现原理是通过向/usr/bin/mipns-xiaomi进程注入代码，劫持mibrain_asr_nlp_record_write函数的调用。
现在没有实现复杂的功能，只是把收到的结果直接输出，还没有写收到之后的处理，所以还不能使用，不过可以测试一下。

如何测试
直接kill掉/usr/bin/mipns-xiaomi这个进程，自己手动在shell中启动，然后执行[url=]get_reponse，之后回车退出。[/url] [url=]libmy.so这个文件和get_reponse放到一个目录中(可能是/data/这个目录)。[/url]
再使用小爱的时候就会在控制台看到输出结果了。

存在问题
1 动态库中的函数还无法正确调用正常的mibrain_asr_nlp_record_write，导致我测试的时候程序直接崩溃了一次，暂时还不知道具体原因。
2 注入的时候使用的都是绝对地址，版本不一致可能无法使用，我使用的版本是1.58.15

get_reponse运行结果

root@mico:/data/get# ./get_response 
targetpid is :2937
addr = -152924160
remote_func_addr:0xf6e29000
mapbase is 0xf6ef4ebc
[+] Calling mmap in target process.
func_addrs 0xf6ef4ebc
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from mmap, return value=f77e0000, pc=0 
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
addr = -149803008
remote_func_addr:0xf7123000
[+] Get imports: dlopen: f7123a88, dlsym: f7123b74, dlclose: f7123b14, dlerror: f7123e1c
library path = /data/libmy.so
[+] Calling dlopen in target process.
func_addrs 0xf7123a88
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from dlopen, return value=397d40, pc=0 
[+] Calling dlsym in target process.
func_addrs 0xf7123b74
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from dlsym, return value=e531ae45, pc=0 
hook_entry_addr = 0x397d40
[+] Calling my in target process.
func_addrs 0xe531ae45
set regs ok
waitpid
ptrace_call_ok
[+] Target process returned from my, return value=1, pc=0 
addr = -149356544
remote_func_addr:0xf7190000
mytesthook_addr = 0xf71bb1a8
Press enter to dlclose and detach

复制代码

/usr/bin/mipns-xiaomi运行结果

root@mico:~# /usr/bin/mipns-xiaomi -c /usr/share/xiaomi/xaudio_engine.conf -r opus32 -l
this is Lx06  
Hardware PCM card 0 'AML-AXGSOUND' device 3 subdevice 0
Its setup is:
  stream       : CAPTURE
  access       : RW_INTERLEAVED
  format       : S16_LE
  subformat    : STD
  channels     : 8
  rate         : 48000
  exact rate   : 48000 (48000/1)
  msbits       : 16
  buffer_size  : 8192
  period_size  : 512
  period_time  : 10666
  tstamp_mode  : NONE
  tstamp_type  : MONOTONIC
  period_step  : 1
  avail_min    : 512
  period_event : 0
  start_threshold  : 1
  stop_threshold   : 8192
  silence_threshold: 0
  silence_size : 0
  boundary     : 1073741824
  appl_ptr     : 0
  hw_ptr       : 0
this is Lx06 mic detect methods 
this is Lx06 position in paramerers.c 
this is Lx06 position in mvdr.c  
/usr/share/xiaomi/dnn_vad_model/model_dense_1_0_transpose.txt 
dense layer1 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_1_1_transpose.txt 
dense layer1 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_2_0_transpose.txt 
dense layer2 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_2_1_transpose.txt 
dense layer2 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_3_0_transpose.txt 
dense layer3 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_3_1_transpose.txt 
dense layer3 bias initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_4_0_transpose.txt 
dense layer4 weight initialized !
/usr/share/xiaomi/dnn_vad_model/model_dense_4_1_transpose.txt 
dense layer4 bias initialized !
all the model initialized !
the DNN model size is 0 
finish dnn inital
xaudio: dnn wrapper init path /usr/share/xiaomi/dnn_we_model
vad model version is: vad_2.1.0_0
min_voice_length=80, min_sil_length=50
/data/libmy.so
/data/libmy.so
rice wakeup 1-level: frame=24005, start frame=23937, end frame=24005, at 2020-6-28-22-29-24.703729
rice wakeup 2-level: frame=24005, start frame=23939, end frame=24005, at 2020-6-28-22-29-24.839266
{
        "code": 0
}
idle
json_object_from_file: error opening file /data/upnp-disc/devices.json: No such file or directory
{"meta":{"type":"RESULT_NLP","nlp":{},"request_id":"f8c9cba402b6e765caa76bf17d4c6f5a","timestamp":1593354566243},"response":{"status":{"code":200,"error_type":"success"},"answer":[{"domain":"soundboxControl","action":"next","text":"","content":{"to_speak":""},"intention":{"score":0.95,"query":"下一曲","domain":"soundboxControl","action":"next","originAction":"next","func":"play_control.switch.common.next","complete":true,"ignoreSpeak":false,"intent_arbitrator_info":{"predict_result_domain":"soundboxControl","func":"play_control.switch.common.next","l2_domain":"soundboxControl","l2_func":"play_control.switch.common.next","score_domains":{"fuzzySearch":{"score":0.1,"func":"knowledge.fuzzy_search.common.common"},"default":{"score":0.05,"func":""},"music":{"score":0.4,"func":"play_control.switch.common.next"},"soundboxControl":{"score":0.95,"func":"play_control.switch.common.next"}},"arbitrator_level":"","dialog_status":"FINISH","single_result_domain":"soundboxControl","multi_result_domain":"soundboxControl","prejudge_status":"PREJUDGE_NOT_AFFECTED"},"domain_judge":"judge","rewrite_infos":{}}}],"version":"2.2","session_id":"8630dc0ab4a147cfa974600c03ba2726","request_id":"f8c9cba402b6e765caa76bf17d4c6f5a","latency":249,"eid":"0:0:0:0:0:0:0:101:0:0:0:0:0:34:0","instructions":[{"header":{"namespace":"Execution","name":"InstructionControl","id":"c9a0274df05a4c619f1c90dc667425a1","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{"behavior":"INSERT_FRONT"}},{"header":{"namespace":"PlaybackController","name":"Next","id":"93458338ff4e450d83aab6323d63320b","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{}},{"header":{"namespace":"Suggestion","name":"ShowContextSuggestions","id":"7d6c7980440c420498e258bf5cb7fb81","dialog_id":"f8c9cba402b6e765caa76bf17d4c6f5a"},"payload":{"cloud_control":{"show_layout":2,"with_double_quotation":false},"suggestions":[{"query":"开启位置信息","send_query":"开启位置信息","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]},{"query":"进入飞行模式","send_query":"进入飞行模式","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]},{"query":"打开震动","send_query":"打开震动","score":0,"suggest_query_type":"COMMERCIAL","domain":"soundboxControl","pkg_name":"","min_version":0,"tags":["general_context_guidance"]}]}}],"tts_enable":false}}
{
        "code": 0
}
idle

复制代码

主要根据github上的一个代码进行适配修改的，在这个系统里面暂时找不到那个链接了，之后再放上了吧。

zybbq

{:3_41:具体要怎样用呢？

yylwhy

刚刚 快要成效了。。。然后上班了

zimiss

yylwhy 发表于 2020-6-28 19:13
刚刚 快要成效了。。。然后上班了

半成品算是完成了٩( •̀㉨•́ )و get！

anmonkey

感谢楼主分享~

zimiss

[url=]附件[/url]
[url=]https://github.com/Minxin/gothookremote.git

[/url]

david_jts

用 LD_PRELOAD 劫持更方便些。 实测最新的 1.62 无效，要降级才行。