gnh1201/Open-Llama
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix typo

Browse Source
This commit is contained in:
LiangSong 2023-05-04 22:32:15 +08:00
parent 98ffab3a97
commit fbb7997607
1 changed files with 3 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
chat_server.py
View File

@ -2,7 +2,7 @@
Author: LiangSong(sl12160010@gmail.com)
Date: 2023-04-06 22:30:10
LastEditors: LiangSong(sl12160010@gmail.com)
LastEditTime: 2023-05-04 22:28:07
LastEditTime: 2023-05-04 22:32:05
FilePath: /Open-Llama/chat_server.py
Description:
@ -88,10 +88,8 @@ with gr.Blocks() as demo:
context = torch.cat(context, dim=-1)
context = context[:, -1024:]
inputs_len = context.shape[1]
context = context.half().cuda()
pred = model.generate(
input_ids=context, max_new_tokens=1024, do_sample=True
)
context = context.cuda()
pred = model.generate(input_ids=context, max_new_tokens=1024, do_sample=True)
pred = pred[:, inputs_len:]
pred = tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)
logging.warn(pred)