博客 / 詳情

返回

語音識別服務funasr搭建

本文討論語音識別功能,使用的是阿里的開源語音識別項目FunASR,含兩種部署方式,社區windows版和docker容器化部署,windows社區版的可以用於本地開發使用,生產環境建議使用容器版。

1、windows社區版部署

  1.1、環境安裝

    軟件需要Visual Studio 2022 c++環境,如果沒有Visual Studio 2022 c++運行環境,雙擊 VC_redist.x64(2022).exe 安裝 Visual Studio 2022環境下編譯的C++程序運行所需要的庫。

   1.2、下載windows社區軟件包

    https://www.modelscope.cn/models/iic/funasr-runtime-win-cpu-x64/files

    image

    隨便選個版本的下載,這裏選擇的是0.2.0版本

   1.3、下載所需模型

git clone https://www.modelscope.cn/damo/speech_fsmn_vad_zh-cn-16k-common-onnx.git;

git clone https://www.modelscope.cn/damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx.git;

git clone https://www.modelscope.cn/damo/speech_ngram_lm_zh-cn-ai-wesp-fst.git;

git clone https://www.modelscope.cn/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx.git;

git clone https://www.modelscope.cn/thuduj12/fst_itn_zh.git

   1.4、啓動服務

     將上面下載的windows社區軟件包解壓後,打開powershell,進入到解壓後的目錄,執行下面的命令

./funasr-wss-server.exe  
--vad-dir D:/developTest/funasr-runtime-resources/models/speech_fsmn_vad_zh-cn-16k-common-onnx 
--model-dir D:/developTest/funasr-runtime-resources/models/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx 
--lm-dir D:/developTest/funasr-runtime-resources/models/speech_ngram_lm_zh-cn-ai-wesp-fst  
--punc-dir D:/developTest/funasr-runtime-resources/models/punc_ct-transformer_cn-en-common-vocab471067-large-onnx 
--itn-dir D:/developTest/funasr-runtime-resources/models/fst_itn_zh 
--certfile 0

    參數説明:

--model-dir  modelscope model ID 或者 本地模型路徑
--vad-dir  modelscope model ID 或者 本地模型路徑
--punc-dir  modelscope model ID 或者 本地模型路徑
--lm-dir modelscope model ID 或者 本地模型路徑
--itn-dir modelscope model ID 或者 本地模型路徑
--certfile  ssl的證書文件,如果需要關閉ssl,參數設置為0

   1.5、客户端調用

    在windows社區版的解壓目錄下有客户端執行文件funasr-wss-client.exe

./funasr-wss-client.exe --server-ip 127.0.0.1 --port 10095 --wav-path asr_example_zh.wav

    服務默認端口是10095,--wav-path指定音頻文件地址

2、docker容器化部署

   2.1、拉取docker鏡像

docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.6

   2.2、啓動容器

    在宿主機創建模型目錄放置模型,這裏的模型建議手動下載,就用上面的git下載下來,如果使用啓動命令自動下載會很慢很卡。

docker run -p 10095:10095 -it --privileged=true -v D:\developTest\funasr-runtime-resources\models:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.6

    映射容器端口,掛載之前創建的存放模型目錄到容器內部。

   2.3、啓動服務

    進入容器內部,進到FunASR/runtime目錄下

    image

     執行如下命令啓動服務

nohup bash run_server.sh \
  --certfile 0 \
  --vad-dir speech_fsmn_vad_zh-cn-16k-common-onnx \
  --model-dir speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx   \
  --punc-dir punc_ct-transformer_cn-en-common-vocab471067-large-onnx  \
  --lm-dir speech_ngram_lm_zh-cn-ai-wesp-fst \
  --itn-dir fst_itn_zh  > log.txt 2>&1 &

    指定模型目錄,這裏的模型都是事先下載好的,就不需要通過啓動命令下載了,certfile設為0,表示關閉ssl。

3、調用示例

  這裏大致寫了兩種java調用方式一種是通過ProcessBuilder,一種是WebSocketClient,大家可以用來看看。

  • 使用ProcessBuilder,運行上面的客户端執行命令,獲取執行結果
public String localTranslation(MultipartRequest multipartRequest) {
        StringBuffer resultBuffer = new StringBuffer();
        // 需要傳遞給exe程序的參數
        String exePath = "D:\\developTest\\funasr-runtime-resources\\funasr-runtime-win-cpu-x64\\funasr-runtime-win-cpu-x64-v0.2.0\\funasr-wss-client.exe";
        String serveIp = "127.0.0.1"; // 假設你想要設置的IP地址
        String port = "10095";
        File targetFile = null;

        try {
            MultipartFile mFile = multipartRequest.getFile("file");
            File dir = new File("D:\\developTest\\funasr-runtime-resources\\wav");
            if (!dir.exists()) {
                dir.mkdirs();
            }
            targetFile = File.createTempFile("tmp_", ".wav", dir);
            mFile.transferTo(targetFile);

            String wavPath = "D:\\developTest\\funasr-runtime-resources\\wav\\"+ targetFile.getName();
            String[] cmd = new String[]{exePath, "--server-ip", serveIp, "--port", port, "--wav-path", wavPath};
            ProcessBuilder pb = new ProcessBuilder();
            pb.command(cmd);

            Process process = pb.start();
            //超時時間
            int timeoutSeconds = 30;//超時30秒自動斷開

            //創建單線程線程池
            ExecutorService executor = Executors.newSingleThreadExecutor();
            Future<?> future = executor.submit(() -> {
                try {
                    pb.redirectErrorStream(true);
                    // 讀取外部程序的輸出
                    BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
                    String line;
                    while ((line = reader.readLine()) != null) {
                        System.out.println(line);
                        resultBuffer.append(line);
                    }

                    // 處理錯誤輸出
                    BufferedReader errorReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
                    while ((line = errorReader.readLine()) != null) {
                        System.out.println(line);
                        if(line.contains("on_message")){
                            String[] array = line.split("on_message =");
                            resultBuffer.append(array[1]);
                        }
                    }
                    // 等待程序執行完成
                    process.waitFor();
                } catch (Exception e) {
                    e.printStackTrace();
                }finally {
                    if(process.isAlive()){
                        process.destroy();
                    }
                }
            });
            try {
                // 等待進程完成或超時
                future.get(timeoutSeconds, TimeUnit.SECONDS);
                System.out.println("進程在規定時間內完成。");
            } catch (Exception e) {
                System.out.println("超時預警: 進程可能掛起。");
                resultBuffer.append("timeout");
            } finally {
                //關閉連接
                if(process.isAlive()){
                    process.destroy();
                }
                executor.shutdownNow(); // 取消任務並關閉線程池
            }
        } catch (Exception e) {
            e.printStackTrace();
            resultBuffer.append("error");
        }finally {
            if (targetFile.exists()) {
                targetFile.delete();
            }
        }
        System.out.println(resultBuffer.toString());
        return resultBuffer.toString();
    }
View Code
  • 使用WebSocketClient直接調用FunASR服務

  Client工具類

package com.example.demo1.web;

import java.io.*;
import java.net.URI;
import java.util.Map;
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.drafts.Draft;
import org.java_websocket.handshake.ServerHandshake;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class FunasrWsClient extends WebSocketClient {

    private static final Logger logger = LoggerFactory.getLogger(FunasrWsClient.class);

    private boolean iseof = false;
    private static String wavPath;
    private static String mode = "offline";
    private static String strChunkSize = "5,10,5";
    private static int chunkInterval = 10;
    private static int sendChunkSize = 1920;
    private static String hotwords="";
    private static String fsthotwords="";
    private String wavName = "javatest";
    private MyCallBack callBack;

    public FunasrWsClient(URI serverUri,MyCallBack callBack) {
        super(serverUri);
        this.callBack = callBack;
    }

    public FunasrWsClient(URI serverUri,String wavPath,MyCallBack callBack) {
        super(serverUri);
        this.callBack = callBack;
        this.wavPath = wavPath;
    }

    public FunasrWsClient(URI serverUri,String strChunkSize,int chunkInterval,String mode,String hotwords,String wavPath,MyCallBack callBack) {
        super(serverUri);
        this.callBack = callBack;
        this.strChunkSize = strChunkSize;
        this.chunkInterval = chunkInterval;
        this.mode = mode;
        this.fsthotwords = hotwords;
        this.wavPath = wavPath;

        int RATE = 16000;
        String[] chunkList = strChunkSize.split(",");
        int int_chunk_size = 60 * Integer.valueOf(chunkList[1].trim()) / chunkInterval;
        int CHUNK = Integer.valueOf(RATE / 1000 * int_chunk_size);
        this.sendChunkSize = CHUNK * 2;
    }

    public class RecWavThread extends Thread {
        private FunasrWsClient funasrClient;

        public RecWavThread(FunasrWsClient funasrClient) {
            this.funasrClient = funasrClient;
        }

        public void run() {
            this.funasrClient.recWav();
        }
    }

    public FunasrWsClient(URI serverUri, Draft draft) {
        super(serverUri, draft);
    }

    public FunasrWsClient(URI serverURI) {

        super(serverURI);
    }

    public FunasrWsClient(URI serverUri, Map<String, String> httpHeaders) {
        super(serverUri, httpHeaders);
    }

    public void getSslContext(String keyfile, String certfile) {
        // TODO
        return;
    }


    public void sendJson(
        String mode, String strChunkSize, int chunkInterval, String wavName, boolean isSpeaking,String suffix) {
        try {

            JSONObject obj = new JSONObject();
            obj.put("mode", mode);
            JSONArray array = new JSONArray();
            String[] chunkList = strChunkSize.split(",");
            for (int i = 0; i < chunkList.length; i++) {
                array.add(Integer.valueOf(chunkList[i].trim()));
            }

            obj.put("chunk_size", array);
            obj.put("chunk_interval", new Integer(chunkInterval));
            obj.put("wav_name", wavName);

            if(FunasrWsClient.hotwords.trim().length()>0)
            {
                String regex = "\\d+";
                JSONObject jsonitems = new JSONObject();
                String[] items=FunasrWsClient.hotwords.trim().split(" ");
                Pattern pattern = Pattern.compile(regex);
                String tmpWords="";
                for(int i=0;i<items.length;i++)
                {

                    Matcher matcher = pattern.matcher(items[i]);

                    if (matcher.matches()) {

                        jsonitems.put(tmpWords.trim(), items[i].trim());
                        tmpWords="";
                        continue;
                    }
                    tmpWords=tmpWords+items[i]+" ";

                }



                obj.put("hotwords", jsonitems.toString());
            }

            if(suffix.equals("wav")){
                suffix="pcm";
            }
            obj.put("wav_format", suffix);
            if (isSpeaking) {
                obj.put("is_speaking", new Boolean(true));
            } else {
                obj.put("is_speaking", new Boolean(false));
            }
            logger.info("sendJson: " + obj);
            // return;

            send(obj.toString());

            return;
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

 
    public void sendEof() {
        try {
            JSONObject obj = new JSONObject();

            obj.put("is_speaking", new Boolean(false));

            logger.info("sendEof: " + obj);
            // return;

            send(obj.toString());
            iseof = true;
            return;
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

   
    public void recWav() {
        String fileName=FunasrWsClient.wavPath;
        String suffix=fileName.split("\\.")[fileName.split("\\.").length-1];
        sendJson(mode, strChunkSize, chunkInterval, wavName, true,suffix);
        File file = new File(FunasrWsClient.wavPath);

        int chunkSize = sendChunkSize;
        byte[] bytes = new byte[chunkSize];

        int readSize = 0;
        try (FileInputStream fis = new FileInputStream(file)) {
            if (FunasrWsClient.wavPath.endsWith(".wav")) {
                fis.read(bytes, 0, 44); 
            }
            readSize = fis.read(bytes, 0, chunkSize);
            while (readSize > 0) {
                if (readSize == chunkSize) {
                    send(bytes); 

                } else {
                    byte[] tmpBytes = new byte[readSize];
                    for (int i = 0; i < readSize; i++) {
                        tmpBytes[i] = bytes[i];
                    }
                    send(tmpBytes);
                }
                
                if (!mode.equals("offline")) {
                    Thread.sleep(Integer.valueOf(chunkSize / 32));
                }

                readSize = fis.read(bytes, 0, chunkSize);
            }

            if (!mode.equals("offline")) {
                Thread.sleep(2000);
                sendEof();
                Thread.sleep(3000);
                close();
            } else {
                sendEof();
            }

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public void onOpen(ServerHandshake handshakedata) {

        RecWavThread thread = new RecWavThread(this);
        thread.start();
    }

    @Override
    public void onMessage(String message) {
        JSONObject jsonObject = new JSONObject();
        JSONParser jsonParser = new JSONParser();
        logger.info("received: " + message);
        try {
            jsonObject = (JSONObject) jsonParser.parse(message);
            logger.info("text: " + jsonObject.get("text"));
            callBack.callBack(jsonObject.get("text"));
            if(jsonObject.containsKey("timestamp"))
            {
                logger.info("timestamp: " + jsonObject.get("timestamp"));
            }
        } catch (org.json.simple.parser.ParseException e) {
            e.printStackTrace();
        }
        if (iseof && mode.equals("offline") && !jsonObject.containsKey("is_final")) {
            close();
        }

        if (iseof && mode.equals("offline") && jsonObject.containsKey("is_final") && jsonObject.get("is_final").equals("false")) {
            close();
        }
    }

    @Override
    public void onClose(int code, String reason, boolean remote) {

        logger.info(
                "Connection closed by "
                        + (remote ? "remote peer" : "us")
                        + " Code: "
                        + code
                        + " Reason: "
                        + reason);
    }

    @Override
    public void onError(Exception ex) {
        logger.info("ex: " + ex);
        ex.printStackTrace();
    }

}
View Code
  調用工具類
public static void main(String[] args) throws URISyntaxException {
    String srvIp = "localhost";
    String srvPort = "10095";
    String wavPath = "D:\\developTest\\funasr-runtime-resources\\wav\\tmp_84677349854990998.wav";
    Object lock = new Object();

    StringBuffer text = new StringBuffer();
    ExecutorService executor = Executors.newSingleThreadExecutor();
    Future<?> future = executor.submit(()->{
        try {
            String wsAddress = "ws://" + srvIp + ":" + srvPort;

            FunasrWsClient c = new FunasrWsClient(new URI(wsAddress),wavPath,new MyCallBack(){
                @Override
                public void callBack(Object obj){
                    text.append(obj.toString());
                    synchronized (lock){
                        try {
                            lock.notify();
                        }catch (Exception e){
                            e.printStackTrace();
                        }
                    }
                }
            });

            synchronized (lock){
                c.connect();
                lock.wait();
            }
        }catch (Exception e){
            // e.printStackTrace();
        }
    });
    try {
        future.get(10, TimeUnit.SECONDS);
        System.out.println("規定時間內完成");
    }catch (Exception e){
        // e.printStackTrace();
        System.out.println("任務超時");
        text.append("任務超時");
    }finally {
        executor.shutdownNow(); // 取消任務並關閉線程池
    }
    System.out.println(text.toString());
}
View Code

 

 

 

user avatar
0 位用戶收藏了這個故事!

發佈 評論

Some HTML is okay.