Woosuk Kwon
|
0deacbce6e
|
Implement single_query_cached_kv_attention kernel (#3)
|
2023-03-01 15:02:19 -08:00 |
|
Woosuk Kwon
|
6aef2278f4
|
[Minor] Fix printing format
|
2023-02-24 11:56:06 +00:00 |
|
Woosuk Kwon
|
1132fae0ca
|
Add Frontend
|
2023-02-24 11:46:43 +00:00 |
|
Woosuk Kwon
|
ef6098ec51
|
Merge pre_step and step
|
2023-02-24 10:36:08 +00:00 |
|
Woosuk Kwon
|
53f70e7334
|
Reduce the number of states in scheduler
|
2023-02-24 10:22:39 +00:00 |
|
Woosuk Kwon
|
afdbe5d373
|
[WIP] Add server script
|
2023-02-24 01:33:37 +00:00 |
|
Woosuk Kwon
|
fdd0f2f472
|
Minor
|
2023-02-23 20:23:47 +00:00 |
|
Woosuk Kwon
|
4f6f4967f6
|
Add get_block_table
|
2023-02-23 07:55:14 +00:00 |
|
Woosuk Kwon
|
331fa0b042
|
Implement scheduler.step & Add a threshold for batch size
|
2023-02-23 07:54:20 +00:00 |
|
Woosuk Kwon
|
2729087efe
|
Fix a ref count bug in swapping
|
2023-02-14 09:20:12 +00:00 |
|
Woosuk Kwon
|
c128c2ed03
|
Fix double free
|
2023-02-14 09:14:05 +00:00 |
|
Woosuk Kwon
|
c80ed212a1
|
Minor
|
2023-02-14 09:06:27 +00:00 |
|
Woosuk Kwon
|
263e91b386
|
Minor
|
2023-02-14 09:05:39 +00:00 |
|
Woosuk Kwon
|
be1e2163c9
|
Fix memory leak in swapping
|
2023-02-14 08:46:33 +00:00 |
|
Woosuk Kwon
|
7e5f604e68
|
Fix bugs in scheduler
|
2023-02-14 02:25:32 +00:00 |
|
Woosuk Kwon
|
1f739f9b03
|
Fix a bug in swap_in and swap_out
|
2023-02-14 02:12:58 +00:00 |
|
Woosuk Kwon
|
b1644f764f
|
Fix can_swap_in
|
2023-02-14 01:57:40 +00:00 |
|
Woosuk Kwon
|
531e1c74e8
|
Fix typo
|
2023-02-13 18:51:33 +00:00 |
|
Woosuk Kwon
|
d3d317665b
|
Fix scheduler
|
2023-02-13 09:37:00 +00:00 |
|
Woosuk Kwon
|
5a309bb588
|
Add scheduler
|
2023-02-13 02:39:53 +00:00 |
|
Woosuk Kwon
|
eb52db1bea
|
Fix can_swap_in
|
2023-02-13 02:38:50 +00:00 |
|
Woosuk Kwon
|
a2a9869cb7
|
SERVING -> RUNNING
|
2023-02-12 08:25:05 +00:00 |
|
Woosuk Kwon
|
46958cf941
|
BlockAllocator -> BlockManager
|
2023-02-12 08:15:18 +00:00 |
|
Woosuk Kwon
|
38ed06a846
|
Minor
|
2023-02-09 11:30:55 +00:00 |
|
Woosuk Kwon
|
aa78aeaa0f
|
Add block manager
|
2023-02-09 11:27:33 +00:00 |
|