pztrn/featurer
1
0
Fork 0
forked from apps/featurer
Code Pull Requests Activity

Add vendor to improve building speed.

Browse Source 
This also adds ability to be built in network-constrained environment.
This commit is contained in:
Stanislav Nikitin
2024-10-12 23:08:41 +05:00
parent 2ecfe7f8ac
commit c49251db31
1603 changed files with 863073 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

201
vendor/github.com/bytedance/sonic/loader/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

245
vendor/github.com/bytedance/sonic/loader/funcdata.go generated vendored Normal file
View File

@@ -0,0 +1,245 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`encoding`
`encoding/binary`
`fmt`
`reflect`
`strings`
`sync`
`unsafe`
)
const (
_MinLC uint8 = 1
_PtrSize uint8 = 8
)
const (
_N_FUNCDATA = 8
_INVALID_FUNCDATA_OFFSET = ^uint32(0)
_FUNC_SIZE = unsafe.Sizeof(_func{})
_MINFUNC = 16 // minimum size for a function
_BUCKETSIZE = 256 * _MINFUNC
_SUBBUCKETS = 16
_SUB_BUCKETSIZE = _BUCKETSIZE / _SUBBUCKETS
)
// Note: This list must match the list in runtime/symtab.go.
const (
FuncFlag_TOPFRAME = 1 << iota
FuncFlag_SPWRITE
FuncFlag_ASM
)
// PCDATA and FUNCDATA table indexes.
//
// See funcdata.h and $GROOT/src/cmd/internal/objabi/funcdata.go.
const (
_FUNCDATA_ArgsPointerMaps = 0
_FUNCDATA_LocalsPointerMaps = 1
_FUNCDATA_StackObjects = 2
_FUNCDATA_InlTree = 3
_FUNCDATA_OpenCodedDeferInfo = 4
_FUNCDATA_ArgInfo = 5
_FUNCDATA_ArgLiveInfo = 6
_FUNCDATA_WrapInfo = 7
// ArgsSizeUnknown is set in Func.argsize to mark all functions
// whose argument size is unknown (C vararg functions, and
// assembly code without an explicit specification).
// This value is generated by the compiler, assembler, or linker.
ArgsSizeUnknown = -0x80000000
)
// moduledata used to cache the funcdata and findfuncbucket of one module
var moduleCache = struct {
m map[*moduledata][]byte
sync.Mutex
}{
m: make(map[*moduledata][]byte),
}
// Func contains information about a function.
type Func struct {
ID uint8 // see runtime/symtab.go
Flag uint8 // see runtime/symtab.go
ArgsSize int32 // args byte size
EntryOff uint32 // start pc, offset to moduledata.text
TextSize uint32 // size of func text
DeferReturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
FileIndex uint32 // index into filetab
Name string // name of function
// PC data
Pcsp *Pcdata // PC -> SP delta
Pcfile *Pcdata // PC -> file index
Pcline *Pcdata // PC -> line number
PcUnsafePoint *Pcdata // PC -> unsafe point, must be PCDATA_UnsafePointSafe or PCDATA_UnsafePointUnsafe
PcStackMapIndex *Pcdata // PC -> stack map index, relative to ArgsPointerMaps and LocalsPointerMaps
PcInlTreeIndex *Pcdata // PC -> inlining tree index, relative to InlTree
PcArgLiveIndex *Pcdata // PC -> arg live index, relative to ArgLiveInfo
// Func data, must implement encoding.BinaryMarshaler
ArgsPointerMaps encoding.BinaryMarshaler // concrete type: *StackMap
LocalsPointerMaps encoding.BinaryMarshaler // concrete type: *StackMap
StackObjects encoding.BinaryMarshaler
InlTree encoding.BinaryMarshaler
OpenCodedDeferInfo encoding.BinaryMarshaler
ArgInfo encoding.BinaryMarshaler
ArgLiveInfo encoding.BinaryMarshaler
WrapInfo encoding.BinaryMarshaler
}
func getOffsetOf(data interface{}, field string) uintptr {
t := reflect.TypeOf(data)
fv, ok := t.FieldByName(field)
if !ok {
panic(fmt.Sprintf("field %s not found in struct %s", field, t.Name()))
}
return fv.Offset
}
func rnd(v int64, r int64) int64 {
if r <= 0 {
return v
}
v += r - 1
c := v % r
if c < 0 {
c += r
}
v -= c
return v
}
var (
byteOrder binary.ByteOrder = binary.LittleEndian
)
func funcNameParts(name string) (string, string, string) {
i := strings.IndexByte(name, '[')
if i < 0 {
return name, "", ""
}
// TODO: use LastIndexByte once the bootstrap compiler is >= Go 1.5.
j := len(name) - 1
for j > i && name[j] != ']' {
j--
}
if j <= i {
return name, "", ""
}
return name[:i], "[...]", name[j+1:]
}
// func name table format:
// nameOff[0] -> namePartA namePartB namePartC \x00
// nameOff[1] -> namePartA namePartB namePartC \x00
// ...
func makeFuncnameTab(funcs []Func) (tab []byte, offs []int32) {
offs = make([]int32, len(funcs))
offset := 1
tab = []byte{0}
for i, f := range funcs {
offs[i] = int32(offset)
a, b, c := funcNameParts(f.Name)
tab = append(tab, a...)
tab = append(tab, b...)
tab = append(tab, c...)
tab = append(tab, 0)
offset += len(a) + len(b) + len(c) + 1
}
return
}
// CU table format:
// cuOffsets[0] -> filetabOffset[0] filetabOffset[1] ... filetabOffset[len(CUs[0].fileNames)-1]
// cuOffsets[1] -> filetabOffset[len(CUs[0].fileNames)] ... filetabOffset[len(CUs[0].fileNames) + len(CUs[1].fileNames)-1]
// ...
//
// file name table format:
// filetabOffset[0] -> CUs[0].fileNames[0] \x00
// ...
// filetabOffset[len(CUs[0]-1)] -> CUs[0].fileNames[len(CUs[0].fileNames)-1] \x00
// ...
// filetabOffset[SUM(CUs,fileNames)-1] -> CUs[len(CU)-1].fileNames[len(CUs[len(CU)-1].fileNames)-1] \x00
func makeFilenametab(cus []compilationUnit) (cutab []uint32, filetab []byte, cuOffsets []uint32) {
cuOffsets = make([]uint32, len(cus))
cuOffset := 0
fileOffset := 0
for i, cu := range cus {
cuOffsets[i] = uint32(cuOffset)
for _, name := range cu.fileNames {
cutab = append(cutab, uint32(fileOffset))
fileOffset += len(name) + 1
filetab = append(filetab, name...)
filetab = append(filetab, 0)
}
cuOffset += len(cu.fileNames)
}
return
}
func writeFuncdata(out *[]byte, funcs []Func) (fstart int, funcdataOffs [][]uint32) {
fstart = len(*out)
*out = append(*out, byte(0))
offs := uint32(1)
funcdataOffs = make([][]uint32, len(funcs))
for i, f := range funcs {
var writer = func(fd encoding.BinaryMarshaler) {
var ab []byte
var err error
if fd != nil {
ab, err = fd.MarshalBinary()
if err != nil {
panic(err)
}
funcdataOffs[i] = append(funcdataOffs[i], offs)
} else {
ab = []byte{0}
funcdataOffs[i] = append(funcdataOffs[i], _INVALID_FUNCDATA_OFFSET)
}
*out = append(*out, ab...)
offs += uint32(len(ab))
}
writer(f.ArgsPointerMaps)
writer(f.LocalsPointerMaps)
writer(f.StackObjects)
writer(f.InlTree)
writer(f.OpenCodedDeferInfo)
writer(f.ArgInfo)
writer(f.ArgLiveInfo)
writer(f.WrapInfo)
}
return
}

460
vendor/github.com/bytedance/sonic/loader/funcdata_compat.go generated vendored Normal file
View File

@@ -0,0 +1,460 @@
//go:build !go1.16
// +build !go1.16
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`os`
`unsafe`
`sort`
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_Magic uint32 = 0xfffffffa
)
type pcHeader struct {
magic uint32 // 0xFFFFFFF0
pad1, pad2 uint8 // 0,0
minLC uint8 // min instruction size
ptrSize uint8 // size of a ptr in bytes
nfunc int // number of functions in the module
nfiles uint // number of entries in the file tab
funcnameOffset uintptr // offset to the funcnametab variable from pcHeader
cuOffset uintptr // offset to the cutab variable from pcHeader
filetabOffset uintptr // offset to the filetab variable from pcHeader
pctabOffset uintptr // offset to the pctab variable from pcHeader
pclnOffset uintptr // offset to the pclntab variable from pcHeader
}
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entry uintptr // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
funcID uint8 // set for certain special runtime functions
_ [2]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}
type funcTab struct {
entry uintptr
funcoff uintptr
}
type bitVector struct {
n int32 // # of bits
bytedata *uint8
}
type ptabEntry struct {
name int32
typ int32
}
type textSection struct {
vaddr uintptr // prelinked section vaddr
end uintptr // vaddr + section length
baseaddr uintptr // relocated section address
}
type modulehash struct {
modulename string
linktimehash string
runtimehash *string
}
// findfuncbucket is an array of these structures.
// Each bucket represents 4096 bytes of the text segment.
// Each subbucket represents 256 bytes of the text segment.
// To find a function given a pc, locate the bucket and subbucket for
// that pc. Add together the idx and subbucket value to obtain a
// function index. Then scan the functab array starting at that
// index to find the target function.
// This table uses 20 bytes for every 4096 bytes of code, or ~0.5% overhead.
type findfuncbucket struct {
idx uint32
_SUBBUCKETS [16]byte
}
type compilationUnit struct {
fileNames []string
}
func makeFtab(funcs []_func, maxpc uintptr) (ftab []funcTab, pclntabSize int64, startLocations []uint32) {
// Allocate space for the pc->func table. This structure consists of a pc offset
// and an offset to the func structure. After that, we have a single pc
// value that marks the end of the last function in the binary.
pclntabSize = int64(len(funcs)*2*int(_PtrSize) + int(_PtrSize))
startLocations = make([]uint32, len(funcs))
for i, f := range funcs {
pclntabSize = rnd(pclntabSize, int64(_PtrSize))
//writePCToFunc
startLocations[i] = uint32(pclntabSize)
pclntabSize += int64(uint8(_FUNC_SIZE) + f.nfuncdata*_PtrSize + uint8(f.npcdata)*4)
}
ftab = make([]funcTab, 0, len(funcs)+1)
// write a map of pc->func info offsets
for i, f := range funcs {
ftab = append(ftab, funcTab{uintptr(f.entry), uintptr(startLocations[i])})
}
// Final entry of table is just end pc offset.
ftab = append(ftab, funcTab{maxpc, 0})
return
}
// Pcln table format: [...]funcTab + [...]_Func
func makePclntable(size int64, startLocations []uint32, funcs []_func, maxpc uintptr, pcdataOffs [][]uint32, funcdataAddr uintptr, funcdataOffs [][]uint32) (pclntab []byte) {
pclntab = make([]byte, size, size)
// write a map of pc->func info offsets
offs := 0
for i, f := range funcs {
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(f.entry))
byteOrder.PutUint64(pclntab[offs+8:offs+16], uint64(startLocations[i]))
offs += 16
}
// Final entry of table is just end pc offset.
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(maxpc))
offs += 8
// write func info table
for i, f := range funcs {
off := startLocations[i]
// write _func structure to pclntab
byteOrder.PutUint64(pclntab[off:off+8], uint64(f.entry))
off += 8
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.nameOff))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.args))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.deferreturn))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcsp))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcfile))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcln))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.npcdata))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.cuOffset))
off += 4
pclntab[off] = f.funcID
// NOTICE: _[2]byte alignment
off += 3
pclntab[off] = f.nfuncdata
off += 1
// NOTICE: _func.pcdata always starts from PcUnsafePoint, which is index 3
for j := 3; j < len(pcdataOffs[i]); j++ {
byteOrder.PutUint32(pclntab[off:off+4], uint32(pcdataOffs[i][j]))
off += 4
}
off = uint32(rnd(int64(off), int64(_PtrSize)))
// funcdata refs as offsets from gofunc
for _, funcdata := range funcdataOffs[i] {
if funcdata == _INVALID_FUNCDATA_OFFSET {
byteOrder.PutUint64(pclntab[off:off+8], 0)
} else {
byteOrder.PutUint64(pclntab[off:off+8], uint64(funcdataAddr)+uint64(funcdata))
}
off += 8
}
}
return
}
// findfunc table used to map pc to belonging func,
// returns the index in the func table.
//
// All text section are divided into buckets sized _BUCKETSIZE(4K):
// every bucket is divided into _SUBBUCKETS sized _SUB_BUCKETSIZE(64),
// and it has a base idx to plus the offset stored in jth subbucket.
// see findfunc() in runtime/symtab.go
func writeFindfunctab(out *[]byte, ftab []funcTab) (start int) {
start = len(*out)
max := ftab[len(ftab)-1].entry
min := ftab[0].entry
nbuckets := (max - min + _BUCKETSIZE - 1) / _BUCKETSIZE
n := (max - min + _SUB_BUCKETSIZE - 1) / _SUB_BUCKETSIZE
tab := make([]findfuncbucket, 0, nbuckets)
var s, e = 0, 0
for i := 0; i<int(nbuckets); i++ {
// store the start func of the bucket
var fb = findfuncbucket{idx: uint32(s)}
// find the last e-th func of the bucket
var pc = min + uintptr((i+1)*_BUCKETSIZE)
for ; e < len(ftab)-1 && ftab[e+1].entry <= pc; e++ {}
for j := 0; j<_SUBBUCKETS && (i*_SUBBUCKETS+j)<int(n); j++ {
// store the start func of the subbucket
fb._SUBBUCKETS[j] = byte(uint32(s) - fb.idx)
// find the s-th end func of the subbucket
pc = min + uintptr(i*_BUCKETSIZE) + uintptr((j+1)*_SUB_BUCKETSIZE)
for ; s < len(ftab)-1 && ftab[s+1].entry <= pc; s++ {}
}
s = e
tab = append(tab, fb)
}
// write findfuncbucket
if len(tab) > 0 {
size := int(unsafe.Sizeof(findfuncbucket{}))*len(tab)
*out = append(*out, rt.BytesFrom(unsafe.Pointer(&tab[0]), size, size)...)
}
return
}
func makeModuledata(name string, filenames []string, funcsp *[]Func, text []byte) (mod *moduledata) {
mod = new(moduledata)
mod.modulename = name
// sort funcs by entry
funcs := *funcsp
sort.Slice(funcs, func(i, j int) bool {
return funcs[i].EntryOff < funcs[j].EntryOff
})
*funcsp = funcs
// make filename table
cu := make([]string, 0, len(filenames))
cu = append(cu, filenames...)
cutab, filetab, cuOffs := makeFilenametab([]compilationUnit{{cu}})
mod.cutab = cutab
mod.filetab = filetab
// make funcname table
funcnametab, nameOffs := makeFuncnameTab(funcs)
mod.funcnametab = funcnametab
// mmap() text and funcdata segements
p := os.Getpagesize()
size := int(rnd(int64(len(text)), int64(p)))
addr := mmap(size)
// copy the machine code
s := rt.BytesFrom(unsafe.Pointer(addr), len(text), size)
copy(s, text)
// make it executable
mprotect(addr, size)
// assign addresses
mod.text = addr
mod.etext = addr + uintptr(size)
mod.minpc = addr
mod.maxpc = addr + uintptr(len(text))
// make pcdata table
// NOTICE: _func only use offset to index pcdata, thus no need mmap() pcdata
cuOff := cuOffs[0]
pctab, pcdataOffs, _funcs := makePctab(funcs, addr, cuOff, nameOffs)
mod.pctab = pctab
// write func data
// NOTICE: _func use mod.gofunc+offset to directly point funcdata, thus need cache funcdata
// TODO: estimate accurate capacity
cache := make([]byte, 0, len(funcs)*int(_PtrSize))
fstart, funcdataOffs := writeFuncdata(&cache, funcs)
// make pc->func (binary search) func table
ftab, pclntSize, startLocations := makeFtab(_funcs, mod.maxpc)
mod.ftab = ftab
// write pc->func (modmap) findfunc table
ffstart := writeFindfunctab(&cache, ftab)
// cache funcdata and findfuncbucket
moduleCache.Lock()
moduleCache.m[mod] = cache
moduleCache.Unlock()
mod.findfunctab = uintptr(rt.IndexByte(cache, ffstart))
funcdataAddr := uintptr(rt.IndexByte(cache, fstart))
// make pclnt table
pclntab := makePclntable(pclntSize, startLocations, _funcs, mod.maxpc, pcdataOffs, funcdataAddr, funcdataOffs)
mod.pclntable = pclntab
// make pc header
mod.pcHeader = &pcHeader {
magic : _Magic,
minLC : _MinLC,
ptrSize : _PtrSize,
nfunc : len(funcs),
nfiles: uint(len(cu)),
funcnameOffset: getOffsetOf(moduledata{}, "funcnametab"),
cuOffset: getOffsetOf(moduledata{}, "cutab"),
filetabOffset: getOffsetOf(moduledata{}, "filetab"),
pctabOffset: getOffsetOf(moduledata{}, "pctab"),
pclnOffset: getOffsetOf(moduledata{}, "pclntable"),
}
// sepecial case: gcdata and gcbss must by non-empty
mod.gcdata = uintptr(unsafe.Pointer(&emptyByte))
mod.gcbss = uintptr(unsafe.Pointer(&emptyByte))
return
}
// makePctab generates pcdelta->valuedelta tables for functions,
// and returns the table and the entry offset of every kind pcdata in the table.
func makePctab(funcs []Func, addr uintptr, cuOffset uint32, nameOffset []int32) (pctab []byte, pcdataOffs [][]uint32, _funcs []_func) {
_funcs = make([]_func, len(funcs))
// Pctab offsets of 0 are considered invalid in the runtime. We respect
// that by just padding a single byte at the beginning of runtime.pctab,
// that way no real offsets can be zero.
pctab = make([]byte, 1, 12*len(funcs)+1)
pcdataOffs = make([][]uint32, len(funcs))
for i, f := range funcs {
_f := &_funcs[i]
var writer = func(pc *Pcdata) {
var ab []byte
var err error
if pc != nil {
ab, err = pc.MarshalBinary()
if err != nil {
panic(err)
}
pcdataOffs[i] = append(pcdataOffs[i], uint32(len(pctab)))
} else {
ab = []byte{0}
pcdataOffs[i] = append(pcdataOffs[i], _PCDATA_INVALID_OFFSET)
}
pctab = append(pctab, ab...)
}
if f.Pcsp != nil {
_f.pcsp = uint32(len(pctab))
}
writer(f.Pcsp)
if f.Pcfile != nil {
_f.pcfile = uint32(len(pctab))
}
writer(f.Pcfile)
if f.Pcline != nil {
_f.pcln = uint32(len(pctab))
}
writer(f.Pcline)
writer(f.PcUnsafePoint)
writer(f.PcStackMapIndex)
writer(f.PcInlTreeIndex)
writer(f.PcArgLiveIndex)
_f.entry = addr + uintptr(f.EntryOff)
_f.nameOff = nameOffset[i]
_f.args = f.ArgsSize
_f.deferreturn = f.DeferReturn
// NOTICE: _func.pcdata is always as [PCDATA_UnsafePoint(0) : PCDATA_ArgLiveIndex(3)]
_f.npcdata = uint32(_N_PCDATA)
_f.cuOffset = cuOffset
_f.funcID = f.ID
_f.nfuncdata = uint8(_N_FUNCDATA)
}
return
}
func registerFunction(name string, pc uintptr, textSize uintptr, fp int, args int, size uintptr, argptrs uintptr, localptrs uintptr) {}

461
vendor/github.com/bytedance/sonic/loader/funcdata_go116.go generated vendored Normal file
View File

@@ -0,0 +1,461 @@
//go:build go1.16 && !go1.18
// +build go1.16,!go1.18
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`os`
`unsafe`
`sort`
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_Magic uint32 = 0xfffffffa
)
type pcHeader struct {
magic uint32 // 0xFFFFFFF0
pad1, pad2 uint8 // 0,0
minLC uint8 // min instruction size
ptrSize uint8 // size of a ptr in bytes
nfunc int // number of functions in the module
nfiles uint // number of entries in the file tab
funcnameOffset uintptr // offset to the funcnametab variable from pcHeader
cuOffset uintptr // offset to the cutab variable from pcHeader
filetabOffset uintptr // offset to the filetab variable from pcHeader
pctabOffset uintptr // offset to the pctab variable from pcHeader
pclnOffset uintptr // offset to the pclntab variable from pcHeader
}
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entry uintptr // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
funcID uint8 // set for certain special runtime functions
_ [2]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}
type funcTab struct {
entry uintptr
funcoff uintptr
}
type bitVector struct {
n int32 // # of bits
bytedata *uint8
}
type ptabEntry struct {
name int32
typ int32
}
type textSection struct {
vaddr uintptr // prelinked section vaddr
end uintptr // vaddr + section length
baseaddr uintptr // relocated section address
}
type modulehash struct {
modulename string
linktimehash string
runtimehash *string
}
// findfuncbucket is an array of these structures.
// Each bucket represents 4096 bytes of the text segment.
// Each subbucket represents 256 bytes of the text segment.
// To find a function given a pc, locate the bucket and subbucket for
// that pc. Add together the idx and subbucket value to obtain a
// function index. Then scan the functab array starting at that
// index to find the target function.
// This table uses 20 bytes for every 4096 bytes of code, or ~0.5% overhead.
type findfuncbucket struct {
idx uint32
_SUBBUCKETS [16]byte
}
type compilationUnit struct {
fileNames []string
}
func makeFtab(funcs []_func, maxpc uintptr) (ftab []funcTab, pclntabSize int64, startLocations []uint32) {
// Allocate space for the pc->func table. This structure consists of a pc offset
// and an offset to the func structure. After that, we have a single pc
// value that marks the end of the last function in the binary.
pclntabSize = int64(len(funcs)*2*int(_PtrSize) + int(_PtrSize))
startLocations = make([]uint32, len(funcs))
for i, f := range funcs {
pclntabSize = rnd(pclntabSize, int64(_PtrSize))
//writePCToFunc
startLocations[i] = uint32(pclntabSize)
pclntabSize += int64(uint8(_FUNC_SIZE) + f.nfuncdata*_PtrSize + uint8(f.npcdata)*4)
}
ftab = make([]funcTab, 0, len(funcs)+1)
// write a map of pc->func info offsets
for i, f := range funcs {
ftab = append(ftab, funcTab{uintptr(f.entry), uintptr(startLocations[i])})
}
// Final entry of table is just end pc offset.
ftab = append(ftab, funcTab{maxpc, 0})
return
}
// Pcln table format: [...]funcTab + [...]_Func
func makePclntable(size int64, startLocations []uint32, funcs []_func, maxpc uintptr, pcdataOffs [][]uint32, funcdataAddr uintptr, funcdataOffs [][]uint32) (pclntab []byte) {
pclntab = make([]byte, size, size)
// write a map of pc->func info offsets
offs := 0
for i, f := range funcs {
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(f.entry))
byteOrder.PutUint64(pclntab[offs+8:offs+16], uint64(startLocations[i]))
offs += 16
}
// Final entry of table is just end pc offset.
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(maxpc))
offs += 8
// write func info table
for i, f := range funcs {
off := startLocations[i]
// write _func structure to pclntab
byteOrder.PutUint64(pclntab[off:off+8], uint64(f.entry))
off += 8
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.nameOff))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.args))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.deferreturn))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcsp))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcfile))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcln))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.npcdata))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.cuOffset))
off += 4
pclntab[off] = f.funcID
// NOTICE: _[2]byte alignment
off += 3
pclntab[off] = f.nfuncdata
off += 1
// NOTICE: _func.pcdata always starts from PcUnsafePoint, which is index 3
for j := 3; j < len(pcdataOffs[i]); j++ {
byteOrder.PutUint32(pclntab[off:off+4], uint32(pcdataOffs[i][j]))
off += 4
}
off = uint32(rnd(int64(off), int64(_PtrSize)))
// funcdata refs as offsets from gofunc
for _, funcdata := range funcdataOffs[i] {
if funcdata == _INVALID_FUNCDATA_OFFSET {
byteOrder.PutUint64(pclntab[off:off+8], 0)
} else {
byteOrder.PutUint64(pclntab[off:off+8], uint64(funcdataAddr)+uint64(funcdata))
}
off += 8
}
}
return
}
// findfunc table used to map pc to belonging func,
// returns the index in the func table.
//
// All text section are divided into buckets sized _BUCKETSIZE(4K):
// every bucket is divided into _SUBBUCKETS sized _SUB_BUCKETSIZE(64),
// and it has a base idx to plus the offset stored in jth subbucket.
// see findfunc() in runtime/symtab.go
func writeFindfunctab(out *[]byte, ftab []funcTab) (start int) {
start = len(*out)
max := ftab[len(ftab)-1].entry
min := ftab[0].entry
nbuckets := (max - min + _BUCKETSIZE - 1) / _BUCKETSIZE
n := (max - min + _SUB_BUCKETSIZE - 1) / _SUB_BUCKETSIZE
tab := make([]findfuncbucket, 0, nbuckets)
var s, e = 0, 0
for i := 0; i<int(nbuckets); i++ {
// store the start func of the bucket
var fb = findfuncbucket{idx: uint32(s)}
// find the last e-th func of the bucket
var pc = min + uintptr((i+1)*_BUCKETSIZE)
for ; e < len(ftab)-1 && ftab[e+1].entry <= pc; e++ {}
for j := 0; j<_SUBBUCKETS && (i*_SUBBUCKETS+j)<int(n); j++ {
// store the start func of the subbucket
fb._SUBBUCKETS[j] = byte(uint32(s) - fb.idx)
// find the s-th end func of the subbucket
pc = min + uintptr(i*_BUCKETSIZE) + uintptr((j+1)*_SUB_BUCKETSIZE)
for ; s < len(ftab)-1 && ftab[s+1].entry <= pc; s++ {}
}
s = e
tab = append(tab, fb)
}
// write findfuncbucket
if len(tab) > 0 {
size := int(unsafe.Sizeof(findfuncbucket{}))*len(tab)
*out = append(*out, rt.BytesFrom(unsafe.Pointer(&tab[0]), size, size)...)
}
return
}
func makeModuledata(name string, filenames []string, funcsp *[]Func, text []byte) (mod *moduledata) {
mod = new(moduledata)
mod.modulename = name
// sort funcs by entry
funcs := *funcsp
sort.Slice(funcs, func(i, j int) bool {
return funcs[i].EntryOff < funcs[j].EntryOff
})
*funcsp = funcs
// make filename table
cu := make([]string, 0, len(filenames))
cu = append(cu, filenames...)
cutab, filetab, cuOffs := makeFilenametab([]compilationUnit{{cu}})
mod.cutab = cutab
mod.filetab = filetab
// make funcname table
funcnametab, nameOffs := makeFuncnameTab(funcs)
mod.funcnametab = funcnametab
// mmap() text and funcdata segements
p := os.Getpagesize()
size := int(rnd(int64(len(text)), int64(p)))
addr := mmap(size)
// copy the machine code
s := rt.BytesFrom(unsafe.Pointer(addr), len(text), size)
copy(s, text)
// make it executable
mprotect(addr, size)
// assign addresses
mod.text = addr
mod.etext = addr + uintptr(size)
mod.minpc = addr
mod.maxpc = addr + uintptr(len(text))
// make pcdata table
// NOTICE: _func only use offset to index pcdata, thus no need mmap() pcdata
cuOff := cuOffs[0]
pctab, pcdataOffs, _funcs := makePctab(funcs, addr, cuOff, nameOffs)
mod.pctab = pctab
// write func data
// NOTICE: _func use mod.gofunc+offset to directly point funcdata, thus need cache funcdata
// TODO: estimate accurate capacity
cache := make([]byte, 0, len(funcs)*int(_PtrSize))
fstart, funcdataOffs := writeFuncdata(&cache, funcs)
// make pc->func (binary search) func table
ftab, pclntSize, startLocations := makeFtab(_funcs, mod.maxpc)
mod.ftab = ftab
// write pc->func (modmap) findfunc table
ffstart := writeFindfunctab(&cache, ftab)
// cache funcdata and findfuncbucket
moduleCache.Lock()
moduleCache.m[mod] = cache
moduleCache.Unlock()
mod.findfunctab = uintptr(rt.IndexByte(cache, ffstart))
funcdataAddr := uintptr(rt.IndexByte(cache, fstart))
// make pclnt table
pclntab := makePclntable(pclntSize, startLocations, _funcs, mod.maxpc, pcdataOffs, funcdataAddr, funcdataOffs)
mod.pclntable = pclntab
// make pc header
mod.pcHeader = &pcHeader {
magic : _Magic,
minLC : _MinLC,
ptrSize : _PtrSize,
nfunc : len(funcs),
nfiles: uint(len(cu)),
funcnameOffset: getOffsetOf(moduledata{}, "funcnametab"),
cuOffset: getOffsetOf(moduledata{}, "cutab"),
filetabOffset: getOffsetOf(moduledata{}, "filetab"),
pctabOffset: getOffsetOf(moduledata{}, "pctab"),
pclnOffset: getOffsetOf(moduledata{}, "pclntable"),
}
// sepecial case: gcdata and gcbss must by non-empty
mod.gcdata = uintptr(unsafe.Pointer(&emptyByte))
mod.gcbss = uintptr(unsafe.Pointer(&emptyByte))
return
}
// makePctab generates pcdelta->valuedelta tables for functions,
// and returns the table and the entry offset of every kind pcdata in the table.
func makePctab(funcs []Func, addr uintptr, cuOffset uint32, nameOffset []int32) (pctab []byte, pcdataOffs [][]uint32, _funcs []_func) {
_funcs = make([]_func, len(funcs))
// Pctab offsets of 0 are considered invalid in the runtime. We respect
// that by just padding a single byte at the beginning of runtime.pctab,
// that way no real offsets can be zero.
pctab = make([]byte, 1, 12*len(funcs)+1)
pcdataOffs = make([][]uint32, len(funcs))
for i, f := range funcs {
_f := &_funcs[i]
var writer = func(pc *Pcdata) {
var ab []byte
var err error
if pc != nil {
ab, err = pc.MarshalBinary()
if err != nil {
panic(err)
}
pcdataOffs[i] = append(pcdataOffs[i], uint32(len(pctab)))
} else {
ab = []byte{0}
pcdataOffs[i] = append(pcdataOffs[i], _PCDATA_INVALID_OFFSET)
}
pctab = append(pctab, ab...)
}
if f.Pcsp != nil {
_f.pcsp = uint32(len(pctab))
}
writer(f.Pcsp)
if f.Pcfile != nil {
_f.pcfile = uint32(len(pctab))
}
writer(f.Pcfile)
if f.Pcline != nil {
_f.pcln = uint32(len(pctab))
}
writer(f.Pcline)
writer(f.PcUnsafePoint)
writer(f.PcStackMapIndex)
writer(f.PcInlTreeIndex)
writer(f.PcArgLiveIndex)
_f.entry = addr + uintptr(f.EntryOff)
_f.nameOff = nameOffset[i]
_f.args = f.ArgsSize
_f.deferreturn = f.DeferReturn
// NOTICE: _func.pcdata is always as [PCDATA_UnsafePoint(0) : PCDATA_ArgLiveIndex(3)]
_f.npcdata = uint32(_N_PCDATA)
_f.cuOffset = cuOffset
_f.funcID = f.ID
_f.nfuncdata = uint8(_N_FUNCDATA)
}
return
}
func registerFunction(name string, pc uintptr, textSize uintptr, fp int, args int, size uintptr, argptrs uintptr, localptrs uintptr) {}

113
vendor/github.com/bytedance/sonic/loader/funcdata_go118.go generated vendored Normal file
View File

@@ -0,0 +1,113 @@
// go:build go1.18 && !go1.20
//go:build go1.18 && !go1.20
// +build go1.18,!go1.20
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_Magic uint32 = 0xfffffff0
)
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
rodata uintptr
gofunc uintptr // go.func.* is actual funcinfo object in image
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entryOff uint32 // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
funcID uint8 // set for certain special runtime functions
flag uint8
_ [1]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}

114
vendor/github.com/bytedance/sonic/loader/funcdata_go120.go generated vendored Normal file
View File

@@ -0,0 +1,114 @@
//go:build go1.20 && !go1.21
// +build go1.20,!go1.21
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_Magic uint32 = 0xFFFFFFF1
)
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
covctrs, ecovctrs uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
rodata uintptr
gofunc uintptr // go.func.* is actual funcinfo object in image
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entryOff uint32 // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
startLine int32 // line number of start of function (func keyword/TEXT directive)
funcID uint8 // set for certain special runtime functions
flag uint8
_ [1]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}

119
vendor/github.com/bytedance/sonic/loader/funcdata_go121.go generated vendored Normal file
View File

@@ -0,0 +1,119 @@
//go:build go1.21 && !go1.23
// +build go1.21,!go1.23
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`unsafe`
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_Magic uint32 = 0xFFFFFFF1
)
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
covctrs, ecovctrs uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
rodata uintptr
gofunc uintptr // go.func.* is actual funcinfo object in image
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
// This slice records the initializing tasks that need to be
// done to start up the program. It is built by the linker.
inittasks []unsafe.Pointer
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entryOff uint32 // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
startLine int32 // line number of start of function (func keyword/TEXT directive)
funcID uint8 // set for certain special runtime functions
flag uint8
_ [1]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}

355
vendor/github.com/bytedance/sonic/loader/funcdata_latest.go generated vendored Normal file
View File

@@ -0,0 +1,355 @@
// go:build go1.18 && !go1.23
// +build go1.18,!go1.23
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`os`
`sort`
`unsafe`
`github.com/bytedance/sonic/loader/internal/rt`
)
type funcTab struct {
entry uint32
funcoff uint32
}
type pcHeader struct {
magic uint32 // 0xFFFFFFF0
pad1, pad2 uint8 // 0,0
minLC uint8 // min instruction size
ptrSize uint8 // size of a ptr in bytes
nfunc int // number of functions in the module
nfiles uint // number of entries in the file tab
textStart uintptr // base for function entry PC offsets in this module, equal to moduledata.text
funcnameOffset uintptr // offset to the funcnametab variable from pcHeader
cuOffset uintptr // offset to the cutab variable from pcHeader
filetabOffset uintptr // offset to the filetab variable from pcHeader
pctabOffset uintptr // offset to the pctab variable from pcHeader
pclnOffset uintptr // offset to the pclntab variable from pcHeader
}
type bitVector struct {
n int32 // # of bits
bytedata *uint8
}
type ptabEntry struct {
name int32
typ int32
}
type textSection struct {
vaddr uintptr // prelinked section vaddr
end uintptr // vaddr + section length
baseaddr uintptr // relocated section address
}
type modulehash struct {
modulename string
linktimehash string
runtimehash *string
}
// findfuncbucket is an array of these structures.
// Each bucket represents 4096 bytes of the text segment.
// Each subbucket represents 256 bytes of the text segment.
// To find a function given a pc, locate the bucket and subbucket for
// that pc. Add together the idx and subbucket value to obtain a
// function index. Then scan the functab array starting at that
// index to find the target function.
// This table uses 20 bytes for every 4096 bytes of code, or ~0.5% overhead.
type findfuncbucket struct {
idx uint32
_SUBBUCKETS [16]byte
}
type compilationUnit struct {
fileNames []string
}
func makeFtab(funcs []_func, maxpc uint32) (ftab []funcTab, pclntabSize int64, startLocations []uint32) {
// Allocate space for the pc->func table. This structure consists of a pc offset
// and an offset to the func structure. After that, we have a single pc
// value that marks the end of the last function in the binary.
pclntabSize = int64(len(funcs)*2*int(_PtrSize) + int(_PtrSize))
startLocations = make([]uint32, len(funcs))
for i, f := range funcs {
pclntabSize = rnd(pclntabSize, int64(_PtrSize))
//writePCToFunc
startLocations[i] = uint32(pclntabSize)
pclntabSize += int64(uint8(_FUNC_SIZE)+f.nfuncdata*4+uint8(f.npcdata)*4)
}
ftab = make([]funcTab, 0, len(funcs)+1)
// write a map of pc->func info offsets
for i, f := range funcs {
ftab = append(ftab, funcTab{uint32(f.entryOff), uint32(startLocations[i])})
}
// Final entry of table is just end pc offset.
ftab = append(ftab, funcTab{maxpc, 0})
return
}
// Pcln table format: [...]funcTab + [...]_Func
func makePclntable(size int64, startLocations []uint32, funcs []_func, maxpc uint32, pcdataOffs [][]uint32, funcdataOffs [][]uint32) (pclntab []byte) {
// Allocate space for the pc->func table. This structure consists of a pc offset
// and an offset to the func structure. After that, we have a single pc
// value that marks the end of the last function in the binary.
pclntab = make([]byte, size, size)
// write a map of pc->func info offsets
offs := 0
for i, f := range funcs {
byteOrder.PutUint32(pclntab[offs:offs+4], uint32(f.entryOff))
byteOrder.PutUint32(pclntab[offs+4:offs+8], uint32(startLocations[i]))
offs += 8
}
// Final entry of table is just end pc offset.
byteOrder.PutUint32(pclntab[offs:offs+4], maxpc)
// write func info table
for i := range funcs {
off := startLocations[i]
// write _func structure to pclntab
fb := rt.BytesFrom(unsafe.Pointer(&funcs[i]), int(_FUNC_SIZE), int(_FUNC_SIZE))
copy(pclntab[off:off+uint32(_FUNC_SIZE)], fb)
off += uint32(_FUNC_SIZE)
// NOTICE: _func.pcdata always starts from PcUnsafePoint, which is index 3
for j := 3; j < len(pcdataOffs[i]); j++ {
byteOrder.PutUint32(pclntab[off:off+4], uint32(pcdataOffs[i][j]))
off += 4
}
// funcdata refs as offsets from gofunc
for _, funcdata := range funcdataOffs[i] {
byteOrder.PutUint32(pclntab[off:off+4], uint32(funcdata))
off += 4
}
}
return
}
// findfunc table used to map pc to belonging func,
// returns the index in the func table.
//
// All text section are divided into buckets sized _BUCKETSIZE(4K):
// every bucket is divided into _SUBBUCKETS sized _SUB_BUCKETSIZE(64),
// and it has a base idx to plus the offset stored in jth subbucket.
// see findfunc() in runtime/symtab.go
func writeFindfunctab(out *[]byte, ftab []funcTab) (start int) {
start = len(*out)
max := ftab[len(ftab)-1].entry
min := ftab[0].entry
nbuckets := (max - min + _BUCKETSIZE - 1) / _BUCKETSIZE
n := (max - min + _SUB_BUCKETSIZE - 1) / _SUB_BUCKETSIZE
tab := make([]findfuncbucket, 0, nbuckets)
var s, e = 0, 0
for i := 0; i<int(nbuckets); i++ {
// store the start s-th func of the bucket
var fb = findfuncbucket{idx: uint32(s)}
// find the last e-th func of the bucket
var pc = min + uint32((i+1)*_BUCKETSIZE)
for ; e < len(ftab)-1 && ftab[e+1].entry <= pc; e++ {}
for j := 0; j<_SUBBUCKETS && (i*_SUBBUCKETS+j)<int(n); j++ {
// store the start func of the subbucket
fb._SUBBUCKETS[j] = byte(uint32(s) - fb.idx)
// find the s-th end func of the subbucket
pc = min + uint32(i*_BUCKETSIZE) + uint32((j+1)*_SUB_BUCKETSIZE)
for ; s < len(ftab)-1 && ftab[s+1].entry <= pc; s++ {}
}
s = e
tab = append(tab, fb)
}
// write findfuncbucket
if len(tab) > 0 {
size := int(unsafe.Sizeof(findfuncbucket{}))*len(tab)
*out = append(*out, rt.BytesFrom(unsafe.Pointer(&tab[0]), size, size)...)
}
return
}
func makeModuledata(name string, filenames []string, funcsp *[]Func, text []byte) (mod *moduledata) {
mod = new(moduledata)
mod.modulename = name
// sort funcs by entry
funcs := *funcsp
sort.Slice(funcs, func(i, j int) bool {
return funcs[i].EntryOff < funcs[j].EntryOff
})
*funcsp = funcs
// make filename table
cu := make([]string, 0, len(filenames))
cu = append(cu, filenames...)
cutab, filetab, cuOffs := makeFilenametab([]compilationUnit{{cu}})
mod.cutab = cutab
mod.filetab = filetab
// make funcname table
funcnametab, nameOffs := makeFuncnameTab(funcs)
mod.funcnametab = funcnametab
// mmap() text and funcdata segements
p := os.Getpagesize()
size := int(rnd(int64(len(text)), int64(p)))
addr := mmap(size)
// copy the machine code
s := rt.BytesFrom(unsafe.Pointer(addr), len(text), size)
copy(s, text)
// make it executable
mprotect(addr, size)
// assign addresses
mod.text = addr
mod.etext = addr + uintptr(size)
mod.minpc = addr
mod.maxpc = addr + uintptr(len(text))
// make pcdata table
// NOTICE: _func only use offset to index pcdata, thus no need mmap() pcdata
cuOff := cuOffs[0]
pctab, pcdataOffs, _funcs := makePctab(funcs, cuOff, nameOffs)
mod.pctab = pctab
// write func data
// NOTICE: _func use mod.gofunc+offset to directly point funcdata, thus need cache funcdata
// TODO: estimate accurate capacity
cache := make([]byte, 0, len(funcs)*int(_PtrSize))
fstart, funcdataOffs := writeFuncdata(&cache, funcs)
// make pc->func (binary search) func table
ftab, pclntSize, startLocations := makeFtab(_funcs, uint32(len(text)))
mod.ftab = ftab
// write pc->func (modmap) findfunc table
ffstart := writeFindfunctab(&cache, ftab)
// cache funcdata and findfuncbucket
moduleCache.Lock()
moduleCache.m[mod] = cache
moduleCache.Unlock()
mod.gofunc = uintptr(unsafe.Pointer(&cache[fstart]))
mod.findfunctab = uintptr(unsafe.Pointer(&cache[ffstart]))
// make pclnt table
pclntab := makePclntable(pclntSize, startLocations, _funcs, uint32(len(text)), pcdataOffs, funcdataOffs)
mod.pclntable = pclntab
// make pc header
mod.pcHeader = &pcHeader {
magic : _Magic,
minLC : _MinLC,
ptrSize : _PtrSize,
nfunc : len(funcs),
nfiles: uint(len(cu)),
textStart: mod.text,
funcnameOffset: getOffsetOf(moduledata{}, "funcnametab"),
cuOffset: getOffsetOf(moduledata{}, "cutab"),
filetabOffset: getOffsetOf(moduledata{}, "filetab"),
pctabOffset: getOffsetOf(moduledata{}, "pctab"),
pclnOffset: getOffsetOf(moduledata{}, "pclntable"),
}
// sepecial case: gcdata and gcbss must by non-empty
mod.gcdata = uintptr(unsafe.Pointer(&emptyByte))
mod.gcbss = uintptr(unsafe.Pointer(&emptyByte))
return
}
// makePctab generates pcdelta->valuedelta tables for functions,
// and returns the table and the entry offset of every kind pcdata in the table.
func makePctab(funcs []Func, cuOffset uint32, nameOffset []int32) (pctab []byte, pcdataOffs [][]uint32, _funcs []_func) {
_funcs = make([]_func, len(funcs))
// Pctab offsets of 0 are considered invalid in the runtime. We respect
// that by just padding a single byte at the beginning of runtime.pctab,
// that way no real offsets can be zero.
pctab = make([]byte, 1, 12*len(funcs)+1)
pcdataOffs = make([][]uint32, len(funcs))
for i, f := range funcs {
_f := &_funcs[i]
var writer = func(pc *Pcdata) {
var ab []byte
var err error
if pc != nil {
ab, err = pc.MarshalBinary()
if err != nil {
panic(err)
}
pcdataOffs[i] = append(pcdataOffs[i], uint32(len(pctab)))
} else {
ab = []byte{0}
pcdataOffs[i] = append(pcdataOffs[i], _PCDATA_INVALID_OFFSET)
}
pctab = append(pctab, ab...)
}
if f.Pcsp != nil {
_f.pcsp = uint32(len(pctab))
}
writer(f.Pcsp)
if f.Pcfile != nil {
_f.pcfile = uint32(len(pctab))
}
writer(f.Pcfile)
if f.Pcline != nil {
_f.pcln = uint32(len(pctab))
}
writer(f.Pcline)
writer(f.PcUnsafePoint)
writer(f.PcStackMapIndex)
writer(f.PcInlTreeIndex)
writer(f.PcArgLiveIndex)
_f.entryOff = f.EntryOff
_f.nameOff = nameOffset[i]
_f.args = f.ArgsSize
_f.deferreturn = f.DeferReturn
// NOTICE: _func.pcdata is always as [PCDATA_UnsafePoint(0) : PCDATA_ArgLiveIndex(3)]
_f.npcdata = uint32(_N_PCDATA)
_f.cuOffset = cuOffset
_f.funcID = f.ID
_f.flag = f.Flag
_f.nfuncdata = uint8(_N_FUNCDATA)
}
return
}
func registerFunction(name string, pc uintptr, textSize uintptr, fp int, args int, size uintptr, argptrs uintptr, localptrs uintptr) {}

197
vendor/github.com/bytedance/sonic/loader/internal/abi/abi.go generated vendored Normal file
View File

@@ -0,0 +1,197 @@
/*
* Copyright 2022 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package abi
import (
`fmt`
`reflect`
`sort`
`strings`
`github.com/bytedance/sonic/loader/internal/rt`
)
type FunctionLayout struct {
FP uint32
Args []Parameter
Rets []Parameter
}
func (self FunctionLayout) String() string {
return self.formatFn()
}
func (self FunctionLayout) ArgSize() uint32 {
size := uintptr(0)
for _, arg := range self.Args {
size += arg.Type.Size()
}
return uint32(size)
}
type slot struct {
p bool
m uint32
}
func (self FunctionLayout) StackMap() *rt.StackMap {
var st []slot
var mb rt.StackMapBuilder
/* add arguments */
for _, v := range self.Args {
st = append(st, slot {
m: v.Mem,
p: v.IsPointer,
})
}
/* add stack-passed return values */
for _, v := range self.Rets {
if !v.InRegister {
st = append(st, slot {
m: v.Mem,
p: v.IsPointer,
})
}
}
/* sort by memory offset */
sort.Slice(st, func(i int, j int) bool {
return st[i].m < st[j].m
})
/* add the bits */
for _, v := range st {
mb.AddField(v.p)
}
/* build the stack map */
return mb.Build()
}
func (self FunctionLayout) formatFn() string {
fp := self.FP
return fmt.Sprintf("\n%#04x\nRets:\n%s\nArgs:\n%s", fp, self.formatSeq(self.Rets, &fp), self.formatSeq(self.Args, &fp))
}
func (self FunctionLayout) formatSeq(v []Parameter, fp *uint32) string {
nb := len(v)
mm := make([]string, 0, len(v))
/* convert each part */
for i := nb-1; i >=0; i-- {
*fp -= PtrSize
mm = append(mm, fmt.Sprintf("%#04x %s", *fp, v[i].String()))
}
/* join them together */
return strings.Join(mm, "\n")
}
type Frame struct {
desc *FunctionLayout
locals []bool
ccall bool
}
func NewFrame(desc *FunctionLayout, locals []bool, ccall bool) Frame {
fr := Frame{}
fr.desc = desc
fr.locals = locals
fr.ccall = ccall
return fr
}
func (self *Frame) String() string {
out := self.desc.String()
off := -8
out += fmt.Sprintf("\n%#4x [Return PC]", off)
off -= 8
out += fmt.Sprintf("\n%#4x [RBP]", off)
off -= 8
for _, v := range ReservedRegs(self.ccall) {
out += fmt.Sprintf("\n%#4x [%v]", off, v)
off -= PtrSize
}
for _, b := range self.locals {
out += fmt.Sprintf("\n%#4x [%v]", off, b)
off -= PtrSize
}
return out
}
func (self *Frame) Prev() uint32 {
return self.Size() + PtrSize
}
func (self *Frame) Size() uint32 {
return uint32(self.Offs() + PtrSize)
}
func (self *Frame) Offs() uint32 {
return uint32(len(ReservedRegs(self.ccall)) * PtrSize + len(self.locals)*PtrSize)
}
func (self *Frame) ArgPtrs() *rt.StackMap {
return self.desc.StackMap()
}
func (self *Frame) LocalPtrs() *rt.StackMap {
var m rt.StackMapBuilder
for _, b := range self.locals {
m.AddFields(len(ReservedRegs(self.ccall)), b)
}
return m.Build()
}
func alignUp(n uint32, a int) uint32 {
return (uint32(n) + uint32(a) - 1) &^ (uint32(a) - 1)
}
func isPointer(vt reflect.Type) bool {
switch vt.Kind() {
case reflect.Bool : fallthrough
case reflect.Int : fallthrough
case reflect.Int8 : fallthrough
case reflect.Int16 : fallthrough
case reflect.Int32 : fallthrough
case reflect.Int64 : fallthrough
case reflect.Uint : fallthrough
case reflect.Uint8 : fallthrough
case reflect.Uint16 : fallthrough
case reflect.Uint32 : fallthrough
case reflect.Uint64 : fallthrough
case reflect.Float32 : fallthrough
case reflect.Float64 : fallthrough
case reflect.Uintptr : return false
case reflect.Chan : fallthrough
case reflect.Func : fallthrough
case reflect.Map : fallthrough
case reflect.Ptr : fallthrough
case reflect.UnsafePointer : return true
case reflect.Complex64 : fallthrough
case reflect.Complex128 : fallthrough
case reflect.Array : fallthrough
case reflect.Struct : panic("abi: unsupported types")
default : panic("abi: invalid value type")
}
}

282
vendor/github.com/bytedance/sonic/loader/internal/abi/abi_amd64.go generated vendored Normal file
View File

@@ -0,0 +1,282 @@
/*
* Copyright 2022 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package abi
import (
`fmt`
`reflect`
`unsafe`
. `github.com/cloudwego/iasm/x86_64`
)
const (
PtrSize = 8 // pointer size
PtrAlign = 8 // pointer alignment
)
var iregOrderC = []Register{
RDI,
RSI,
RDX,
RCX,
R8,
R9,
}
var xregOrderC = []Register{
XMM0,
XMM1,
XMM2,
XMM3,
XMM4,
XMM5,
XMM6,
XMM7,
}
var (
intType = reflect.TypeOf(0)
ptrType = reflect.TypeOf(unsafe.Pointer(nil))
)
func (self *Frame) argv(i int) *MemoryOperand {
return Ptr(RSP, int32(self.Prev() + self.desc.Args[i].Mem))
}
// spillv is used for growstack spill registers
func (self *Frame) spillv(i int) *MemoryOperand {
// remain one slot for caller return pc
return Ptr(RSP, PtrSize + int32(self.desc.Args[i].Mem))
}
func (self *Frame) retv(i int) *MemoryOperand {
return Ptr(RSP, int32(self.Prev() + self.desc.Rets[i].Mem))
}
func (self *Frame) resv(i int) *MemoryOperand {
return Ptr(RSP, int32(self.Offs() - uint32((i+1) * PtrSize)))
}
func (self *Frame) emitGrowStack(p *Program, entry *Label) {
// spill all register arguments
for i, v := range self.desc.Args {
if v.InRegister {
if v.IsFloat == floatKind64 {
p.MOVSD(v.Reg, self.spillv(i))
} else if v.IsFloat == floatKind32 {
p.MOVSS(v.Reg, self.spillv(i))
}else {
p.MOVQ(v.Reg, self.spillv(i))
}
}
}
// call runtime.morestack_noctxt
p.MOVQ(F_morestack_noctxt, R12)
p.CALLQ(R12)
// load all register arguments
for i, v := range self.desc.Args {
if v.InRegister {
if v.IsFloat == floatKind64 {
p.MOVSD(self.spillv(i), v.Reg)
} else if v.IsFloat == floatKind32 {
p.MOVSS(self.spillv(i), v.Reg)
}else {
p.MOVQ(self.spillv(i), v.Reg)
}
}
}
// jump back to the function entry
p.JMP(entry)
}
func (self *Frame) GrowStackTextSize() uint32 {
p := DefaultArch.CreateProgram()
// spill all register arguments
for i, v := range self.desc.Args {
if v.InRegister {
if v.IsFloat == floatKind64 {
p.MOVSD(v.Reg, self.spillv(i))
} else if v.IsFloat == floatKind32 {
p.MOVSS(v.Reg, self.spillv(i))
}else {
p.MOVQ(v.Reg, self.spillv(i))
}
}
}
// call runtime.morestack_noctxt
p.MOVQ(F_morestack_noctxt, R12)
p.CALLQ(R12)
// load all register arguments
for i, v := range self.desc.Args {
if v.InRegister {
if v.IsFloat == floatKind64 {
p.MOVSD(self.spillv(i), v.Reg)
} else if v.IsFloat == floatKind32 {
p.MOVSS(self.spillv(i), v.Reg)
} else {
p.MOVQ(self.spillv(i), v.Reg)
}
}
}
// jump back to the function entry
l := CreateLabel("")
p.Link(l)
p.JMP(l)
return uint32(len(p.Assemble(0)))
}
func (self *Frame) emitPrologue(p *Program) {
p.SUBQ(self.Size(), RSP)
p.MOVQ(RBP, Ptr(RSP, int32(self.Offs())))
p.LEAQ(Ptr(RSP, int32(self.Offs())), RBP)
}
func (self *Frame) emitEpilogue(p *Program) {
p.MOVQ(Ptr(RSP, int32(self.Offs())), RBP)
p.ADDQ(self.Size(), RSP)
p.RET()
}
func (self *Frame) emitReserveRegs(p *Program) {
// spill reserved registers
for i, r := range ReservedRegs(self.ccall) {
switch r.(type) {
case Register64:
p.MOVQ(r, self.resv(i))
case XMMRegister:
p.MOVSD(r, self.resv(i))
default:
panic(fmt.Sprintf("unsupported register type %t to reserve", r))
}
}
}
func (self *Frame) emitSpillPtrs(p *Program) {
// spill pointer argument registers
for i, r := range self.desc.Args {
if r.InRegister && r.IsPointer {
p.MOVQ(r.Reg, self.argv(i))
}
}
}
func (self *Frame) emitClearPtrs(p *Program) {
// spill pointer argument registers
for i, r := range self.desc.Args {
if r.InRegister && r.IsPointer {
p.MOVQ(int64(0), self.argv(i))
}
}
}
func (self *Frame) emitCallC(p *Program, addr uintptr) {
p.MOVQ(addr, RAX)
p.CALLQ(RAX)
}
type floatKind uint8
const (
notFloatKind floatKind = iota
floatKind32
floatKind64
)
type Parameter struct {
InRegister bool
IsPointer bool
IsFloat floatKind
Reg Register
Mem uint32
Type reflect.Type
}
func mkIReg(vt reflect.Type, reg Register64) (p Parameter) {
p.Reg = reg
p.Type = vt
p.InRegister = true
p.IsPointer = isPointer(vt)
return
}
func isFloat(vt reflect.Type) floatKind {
switch vt.Kind() {
case reflect.Float32:
return floatKind32
case reflect.Float64:
return floatKind64
default:
return notFloatKind
}
}
func mkXReg(vt reflect.Type, reg XMMRegister) (p Parameter) {
p.Reg = reg
p.Type = vt
p.InRegister = true
p.IsFloat = isFloat(vt)
return
}
func mkStack(vt reflect.Type, mem uint32) (p Parameter) {
p.Mem = mem
p.Type = vt
p.InRegister = false
p.IsPointer = isPointer(vt)
p.IsFloat = isFloat(vt)
return
}
func (self Parameter) String() string {
if self.InRegister {
return fmt.Sprintf("[%%%s, Pointer(%v), Float(%v)]", self.Reg, self.IsPointer, self.IsFloat)
} else {
return fmt.Sprintf("[%d(FP), Pointer(%v), Float(%v)]", self.Mem, self.IsPointer, self.IsFloat)
}
}
func CallC(addr uintptr, fr Frame, maxStack uintptr) []byte {
p := DefaultArch.CreateProgram()
stack := CreateLabel("_stack_grow")
entry := CreateLabel("_entry")
p.Link(entry)
fr.emitStackCheck(p, stack, maxStack)
fr.emitPrologue(p)
fr.emitReserveRegs(p)
fr.emitSpillPtrs(p)
fr.emitExchangeArgs(p)
fr.emitCallC(p, addr)
fr.emitExchangeRets(p)
fr.emitRestoreRegs(p)
fr.emitEpilogue(p)
p.Link(stack)
fr.emitGrowStack(p, entry)
return p.Assemble(0)
}
func (self *Frame) emitDebug(p *Program) {
p.INT(3)
}

182
vendor/github.com/bytedance/sonic/loader/internal/abi/abi_legacy_amd64.go generated vendored Normal file
View File

@@ -0,0 +1,182 @@
//go:build !go1.17
// +build !go1.17
/*
* Copyright 2022 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package abi
import (
`fmt`
`reflect`
`runtime`
. `github.com/cloudwego/iasm/x86_64`
)
func ReservedRegs(callc bool) []Register {
return nil
}
func salloc(p []Parameter, sp uint32, vt reflect.Type) (uint32, []Parameter) {
switch vt.Kind() {
case reflect.Bool : return sp + 8, append(p, mkStack(reflect.TypeOf(false), sp))
case reflect.Int : return sp + 8, append(p, mkStack(intType, sp))
case reflect.Int8 : return sp + 8, append(p, mkStack(reflect.TypeOf(int8(0)), sp))
case reflect.Int16 : return sp + 8, append(p, mkStack(reflect.TypeOf(int16(0)), sp))
case reflect.Int32 : return sp + 8, append(p, mkStack(reflect.TypeOf(int32(0)), sp))
case reflect.Int64 : return sp + 8, append(p, mkStack(reflect.TypeOf(int64(0)), sp))
case reflect.Uint : return sp + 8, append(p, mkStack(reflect.TypeOf(uint(0)), sp))
case reflect.Uint8 : return sp + 8, append(p, mkStack(reflect.TypeOf(uint8(0)), sp))
case reflect.Uint16 : return sp + 8, append(p, mkStack(reflect.TypeOf(uint16(0)), sp))
case reflect.Uint32 : return sp + 8, append(p, mkStack(reflect.TypeOf(uint32(0)), sp))
case reflect.Uint64 : return sp + 8, append(p, mkStack(reflect.TypeOf(uint64(0)), sp))
case reflect.Uintptr : return sp + 8, append(p, mkStack(reflect.TypeOf(uintptr(0)), sp))
case reflect.Float32 : return sp + 8, append(p, mkStack(reflect.TypeOf(float32(0)), sp))
case reflect.Float64 : return sp + 8, append(p, mkStack(reflect.TypeOf(float64(0)), sp))
case reflect.Complex64 : panic("abi: go116: not implemented: complex64")
case reflect.Complex128 : panic("abi: go116: not implemented: complex128")
case reflect.Array : panic("abi: go116: not implemented: arrays")
case reflect.Chan : return sp + 8, append(p, mkStack(reflect.TypeOf((chan int)(nil)), sp))
case reflect.Func : return sp + 8, append(p, mkStack(reflect.TypeOf((func())(nil)), sp))
case reflect.Map : return sp + 8, append(p, mkStack(reflect.TypeOf((map[int]int)(nil)), sp))
case reflect.Ptr : return sp + 8, append(p, mkStack(reflect.TypeOf((*int)(nil)), sp))
case reflect.UnsafePointer : return sp + 8, append(p, mkStack(ptrType, sp))
case reflect.Interface : return sp + 16, append(p, mkStack(ptrType, sp), mkStack(ptrType, sp + 8))
case reflect.Slice : return sp + 24, append(p, mkStack(ptrType, sp), mkStack(intType, sp + 8), mkStack(intType, sp + 16))
case reflect.String : return sp + 16, append(p, mkStack(ptrType, sp), mkStack(intType, sp + 8))
case reflect.Struct : panic("abi: go116: not implemented: structs")
default : panic("abi: invalid value type")
}
}
func NewFunctionLayout(ft reflect.Type) FunctionLayout {
var sp uint32
var fn FunctionLayout
/* assign every arguments */
for i := 0; i < ft.NumIn(); i++ {
sp, fn.Args = salloc(fn.Args, sp, ft.In(i))
}
/* assign every return value */
for i := 0; i < ft.NumOut(); i++ {
sp, fn.Rets = salloc(fn.Rets, sp, ft.Out(i))
}
/* update function ID and stack pointer */
fn.FP = sp
return fn
}
func (self *Frame) emitExchangeArgs(p *Program) {
iregArgs, xregArgs := 0, 0
for _, v := range self.desc.Args {
if v.IsFloat != notFloatKind {
xregArgs += 1
} else {
iregArgs += 1
}
}
if iregArgs > len(iregOrderC) {
panic("too many arguments, only support at most 6 integer arguments now")
}
if xregArgs > len(xregOrderC) {
panic("too many arguments, only support at most 8 float arguments now")
}
ic, xc := iregArgs, xregArgs
for i := 0; i < len(self.desc.Args); i++ {
arg := self.desc.Args[i]
if arg.IsFloat == floatKind64 {
p.MOVSD(self.argv(i), xregOrderC[xregArgs - xc])
xc -= 1
} else if arg.IsFloat == floatKind32 {
p.MOVSS(self.argv(i), xregOrderC[xregArgs - xc])
xc -= 1
} else {
p.MOVQ(self.argv(i), iregOrderC[iregArgs - ic])
ic -= 1
}
}
}
func (self *Frame) emitStackCheck(p *Program, to *Label, maxStack uintptr) {
// get the current goroutine
switch runtime.GOOS {
case "linux" : p.MOVQ(Abs(-8), R14).FS()
case "darwin" : p.MOVQ(Abs(0x30), R14).GS()
case "windows": break // windows always stores G pointer at R14
default : panic("unsupported operating system")
}
// check the stack guard
p.LEAQ(Ptr(RSP, -int32(self.Size() + uint32(maxStack))), RAX)
p.CMPQ(Ptr(R14, _G_stackguard0), RAX)
p.JBE(to)
}
func (self *Frame) StackCheckTextSize() uint32 {
p := DefaultArch.CreateProgram()
// get the current goroutine
switch runtime.GOOS {
case "linux" : p.MOVQ(Abs(-8), R14).FS()
case "darwin" : p.MOVQ(Abs(0x30), R14).GS()
case "windows": break // windows always stores G pointer at R14
default : panic("unsupported operating system")
}
// check the stack guard
p.LEAQ(Ptr(RSP, -int32(self.Size())), RAX)
p.CMPQ(Ptr(R14, _G_stackguard0), RAX)
l := CreateLabel("")
p.Link(l)
p.JBE(l)
return uint32(len(p.Assemble(0)))
}
func (self *Frame) emitExchangeRets(p *Program) {
if len(self.desc.Rets) > 1 {
panic("too many results, only support one result now")
}
// store result
if len(self.desc.Rets) ==1 {
if self.desc.Rets[0].IsFloat == floatKind64 {
p.MOVSD(xregOrderC[0], self.retv(0))
} else if self.desc.Rets[0].IsFloat == floatKind32 {
p.MOVSS(xregOrderC[0], self.retv(0))
} else {
p.MOVQ(RAX, self.retv(0))
}
}
}
func (self *Frame) emitRestoreRegs(p *Program) {
// load reserved registers
for i, r := range ReservedRegs(self.ccall) {
switch r.(type) {
case Register64:
p.MOVQ(self.resv(i), r)
case XMMRegister:
p.MOVSD(self.resv(i), r)
default:
panic(fmt.Sprintf("unsupported register type %t to reserve", r))
}
}
}

316
vendor/github.com/bytedance/sonic/loader/internal/abi/abi_regabi_amd64.go generated vendored Normal file
View File

@@ -0,0 +1,316 @@
//go:build go1.17
// +build go1.17
/*
* Copyright 2022 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/** Go Internal ABI implementation
*
* This module implements the function layout algorithm described by the Go internal ABI.
* See https://github.com/golang/go/blob/master/src/cmd/compile/abi-internal.md for more info.
*/
package abi
import (
`fmt`
`reflect`
. `github.com/cloudwego/iasm/x86_64`
)
/** Frame Structure of the Generated Function
FP +------------------------------+
| . . . |
| 2nd reg argument spill space |
+ 1st reg argument spill space |
| <pointer-sized alignment> |
| . . . |
| 2nd stack-assigned result |
+ 1st stack-assigned result |
| <pointer-sized alignment> |
| . . . |
| 2nd stack-assigned argument |
| 1st stack-assigned argument |
| stack-assigned receiver |
prev() +------------------------------+ (Previous Frame)
Return PC |
size() -------------------------------|
Saved RBP |
offs() -------------------------------|
1th Reserved Registers |
-------------------------------|
2th Reserved Registers |
-------------------------------|
Local Variables |
RSP -------------------------------|↓ lower addresses
*/
const zeroRegGo = XMM15
var iregOrderGo = [...]Register64 {
RAX,// RDI
RBX,// RSI
RCX,// RDX
RDI,// RCX
RSI,// R8
R8, // R9
R9,
R10,
R11,
}
var xregOrderGo = [...]XMMRegister {
XMM0,
XMM1,
XMM2,
XMM3,
XMM4,
XMM5,
XMM6,
XMM7,
XMM8,
XMM9,
XMM10,
XMM11,
XMM12,
XMM13,
XMM14,
}
func ReservedRegs(callc bool) []Register {
if callc {
return nil
}
return []Register {
R14, // current goroutine
R15, // GOT reference
}
}
type stackAlloc struct {
s uint32
i int
x int
}
func (self *stackAlloc) reset() {
self.i, self.x = 0, 0
}
func (self *stackAlloc) ireg(vt reflect.Type) (p Parameter) {
p = mkIReg(vt, iregOrderGo[self.i])
self.i++
return
}
func (self *stackAlloc) xreg(vt reflect.Type) (p Parameter) {
p = mkXReg(vt, xregOrderGo[self.x])
self.x++
return
}
func (self *stackAlloc) stack(vt reflect.Type) (p Parameter) {
p = mkStack(vt, self.s)
self.s += uint32(vt.Size())
return
}
func (self *stackAlloc) spill(n uint32, a int) uint32 {
self.s = alignUp(self.s, a) + n
return self.s
}
func (self *stackAlloc) alloc(p []Parameter, vt reflect.Type) []Parameter {
nb := vt.Size()
vk := vt.Kind()
/* zero-sized objects are allocated on stack */
if nb == 0 {
return append(p, mkStack(intType, self.s))
}
/* check for value type */
switch vk {
case reflect.Bool : return self.valloc(p, reflect.TypeOf(false))
case reflect.Int : return self.valloc(p, intType)
case reflect.Int8 : return self.valloc(p, reflect.TypeOf(int8(0)))
case reflect.Int16 : return self.valloc(p, reflect.TypeOf(int16(0)))
case reflect.Int32 : return self.valloc(p, reflect.TypeOf(uint32(0)))
case reflect.Int64 : return self.valloc(p, reflect.TypeOf(int64(0)))
case reflect.Uint : return self.valloc(p, reflect.TypeOf(uint(0)))
case reflect.Uint8 : return self.valloc(p, reflect.TypeOf(uint8(0)))
case reflect.Uint16 : return self.valloc(p, reflect.TypeOf(uint16(0)))
case reflect.Uint32 : return self.valloc(p, reflect.TypeOf(uint32(0)))
case reflect.Uint64 : return self.valloc(p, reflect.TypeOf(uint64(0)))
case reflect.Uintptr : return self.valloc(p, reflect.TypeOf(uintptr(0)))
case reflect.Float32 : return self.valloc(p, reflect.TypeOf(float32(0)))
case reflect.Float64 : return self.valloc(p, reflect.TypeOf(float64(0)))
case reflect.Complex64 : panic("abi: go117: not implemented: complex64")
case reflect.Complex128 : panic("abi: go117: not implemented: complex128")
case reflect.Array : panic("abi: go117: not implemented: arrays")
case reflect.Chan : return self.valloc(p, reflect.TypeOf((chan int)(nil)))
case reflect.Func : return self.valloc(p, reflect.TypeOf((func())(nil)))
case reflect.Map : return self.valloc(p, reflect.TypeOf((map[int]int)(nil)))
case reflect.Ptr : return self.valloc(p, reflect.TypeOf((*int)(nil)))
case reflect.UnsafePointer : return self.valloc(p, ptrType)
case reflect.Interface : return self.valloc(p, ptrType, ptrType)
case reflect.Slice : return self.valloc(p, ptrType, intType, intType)
case reflect.String : return self.valloc(p, ptrType, intType)
case reflect.Struct : panic("abi: go117: not implemented: structs")
default : panic("abi: invalid value type")
}
}
func (self *stackAlloc) valloc(p []Parameter, vts ...reflect.Type) []Parameter {
for _, vt := range vts {
enum := isFloat(vt)
if enum != notFloatKind && self.x < len(xregOrderGo) {
p = append(p, self.xreg(vt))
} else if enum == notFloatKind && self.i < len(iregOrderGo) {
p = append(p, self.ireg(vt))
} else {
p = append(p, self.stack(vt))
}
}
return p
}
func NewFunctionLayout(ft reflect.Type) FunctionLayout {
var sa stackAlloc
var fn FunctionLayout
/* assign every arguments */
for i := 0; i < ft.NumIn(); i++ {
fn.Args = sa.alloc(fn.Args, ft.In(i))
}
/* reset the register counter, and add a pointer alignment field */
sa.reset()
/* assign every return value */
for i := 0; i < ft.NumOut(); i++ {
fn.Rets = sa.alloc(fn.Rets, ft.Out(i))
}
sa.spill(0, PtrAlign)
/* assign spill slots */
for i := 0; i < len(fn.Args); i++ {
if fn.Args[i].InRegister {
fn.Args[i].Mem = sa.spill(PtrSize, PtrAlign) - PtrSize
}
}
/* add the final pointer alignment field */
fn.FP = sa.spill(0, PtrAlign)
return fn
}
func (self *Frame) emitExchangeArgs(p *Program) {
iregArgs := make([]Parameter, 0, len(self.desc.Args))
xregArgs := 0
for _, v := range self.desc.Args {
if v.InRegister {
if v.IsFloat != notFloatKind {
xregArgs += 1
} else {
iregArgs = append(iregArgs, v)
}
} else {
panic("not support stack-assgined arguments now")
}
}
if xregArgs > len(xregOrderC) {
panic("too many arguments, only support at most 8 integer register arguments now")
}
switch len(iregArgs) {
case 0, 1, 2, 3: {
//Fast-Path: when arguments count are less than four, just exchange the registers
for i := 0; i < len(iregArgs); i++ {
p.MOVQ(iregOrderGo[i], iregOrderC[i])
}
}
case 4, 5, 6: {
// need to spill 3th ~ regArgs registers before exchange
for i := 3; i < len(iregArgs); i++ {
arg := iregArgs[i]
// pointer args have already been spilled
if !arg.IsPointer {
p.MOVQ(iregOrderGo[i], Ptr(RSP, int32(self.Prev() + arg.Mem)))
}
}
p.MOVQ(iregOrderGo[0], iregOrderC[0])
p.MOVQ(iregOrderGo[1], iregOrderC[1])
p.MOVQ(iregOrderGo[2], iregOrderC[2])
for i := 3; i < len(iregArgs); i++ {
arg := iregArgs[i]
p.MOVQ(Ptr(RSP, int32(self.Prev() + arg.Mem)), iregOrderC[i])
}
}
default:
panic("too many arguments, only support at most 6 integer register arguments now")
}
}
func (self *Frame) emitStackCheck(p *Program, to *Label, maxStack uintptr) {
p.LEAQ(Ptr(RSP, int32(-(self.Size() + uint32(maxStack)))), R12)
p.CMPQ(Ptr(R14, _G_stackguard0), R12)
p.JBE(to)
}
func (self *Frame) StackCheckTextSize() uint32 {
p := DefaultArch.CreateProgram()
p.LEAQ(Ptr(RSP, int32(-(self.Size()))), R12)
p.CMPQ(Ptr(R14, _G_stackguard0), R12)
to := CreateLabel("")
p.Link(to)
p.JBE(to)
return uint32(len(p.Assemble(0)))
}
func (self *Frame) emitExchangeRets(p *Program) {
if len(self.desc.Rets) > 1 {
panic("too many results, only support one result now")
}
// store result
if len(self.desc.Rets) == 1 && !self.desc.Rets[0].InRegister {
if self.desc.Rets[0].IsFloat == floatKind64 {
p.MOVSD(xregOrderC[0], self.retv(0))
} else if self.desc.Rets[0].IsFloat == floatKind32 {
p.MOVSS(xregOrderC[0], self.retv(0))
} else {
p.MOVQ(RAX, self.retv(0))
}
}
}
func (self *Frame) emitRestoreRegs(p *Program) {
// load reserved registers
for i, r := range ReservedRegs(self.ccall) {
switch r.(type) {
case Register64:
p.MOVQ(self.resv(i), r)
case XMMRegister:
p.MOVSD(self.resv(i), r)
default:
panic(fmt.Sprintf("unsupported register type %t to reserve", r))
}
}
// zero xmm15 for go abi
p.XORPS(zeroRegGo, zeroRegGo)
}

35
vendor/github.com/bytedance/sonic/loader/internal/abi/stubs.go generated vendored Normal file
View File

@@ -0,0 +1,35 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package abi
import (
_ `unsafe`
`github.com/bytedance/sonic/loader/internal/rt`
)
const (
_G_stackguard0 = 0x10
)
var (
F_morestack_noctxt = uintptr(rt.FuncAddr(morestack_noctxt))
)
//go:linkname morestack_noctxt runtime.morestack_noctxt
func morestack_noctxt()

62
vendor/github.com/bytedance/sonic/loader/internal/rt/fastmem.go generated vendored Normal file
View File

@@ -0,0 +1,62 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package rt
import (
`unsafe`
`reflect`
)
//go:nosplit
func Mem2Str(v []byte) (s string) {
(*GoString)(unsafe.Pointer(&s)).Len = (*GoSlice)(unsafe.Pointer(&v)).Len
(*GoString)(unsafe.Pointer(&s)).Ptr = (*GoSlice)(unsafe.Pointer(&v)).Ptr
return
}
//go:nosplit
func Str2Mem(s string) (v []byte) {
(*GoSlice)(unsafe.Pointer(&v)).Cap = (*GoString)(unsafe.Pointer(&s)).Len
(*GoSlice)(unsafe.Pointer(&v)).Len = (*GoString)(unsafe.Pointer(&s)).Len
(*GoSlice)(unsafe.Pointer(&v)).Ptr = (*GoString)(unsafe.Pointer(&s)).Ptr
return
}
func BytesFrom(p unsafe.Pointer, n int, c int) (r []byte) {
(*GoSlice)(unsafe.Pointer(&r)).Ptr = p
(*GoSlice)(unsafe.Pointer(&r)).Len = n
(*GoSlice)(unsafe.Pointer(&r)).Cap = c
return
}
func FuncAddr(f interface{}) unsafe.Pointer {
if vv := UnpackEface(f); vv.Type.Kind() != reflect.Func {
panic("f is not a function")
} else {
return *(*unsafe.Pointer)(vv.Value)
}
}
//go:nocheckptr
func IndexChar(src string, index int) unsafe.Pointer {
return unsafe.Pointer(uintptr((*GoString)(unsafe.Pointer(&src)).Ptr) + uintptr(index))
}
//go:nocheckptr
func IndexByte(ptr []byte, index int) unsafe.Pointer {
return unsafe.Pointer(uintptr((*GoSlice)(unsafe.Pointer(&ptr)).Ptr) + uintptr(index))
}

183
vendor/github.com/bytedance/sonic/loader/internal/rt/fastvalue.go generated vendored Normal file
View File

@@ -0,0 +1,183 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package rt
import (
`reflect`
`unsafe`
)
var (
reflectRtypeItab = findReflectRtypeItab()
)
// GoType.KindFlags const
const (
F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1
)
// GoType.Flags const
const (
tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3
)
type GoType struct {
Size uintptr
PtrData uintptr
Hash uint32
Flags uint8
Align uint8
FieldAlign uint8
KindFlags uint8
Traits unsafe.Pointer
GCData *byte
Str int32
PtrToSelf int32
}
func (self *GoType) IsNamed() bool {
return (self.Flags & tflagNamed) != 0
}
func (self *GoType) Kind() reflect.Kind {
return reflect.Kind(self.KindFlags & F_kind_mask)
}
func (self *GoType) Pack() (t reflect.Type) {
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return
}
func (self *GoType) String() string {
return self.Pack().String()
}
func (self *GoType) Indirect() bool {
return self.KindFlags & F_direct == 0
}
type GoItab struct {
it unsafe.Pointer
Vt *GoType
hv uint32
_ [4]byte
fn [1]uintptr
}
type GoIface struct {
Itab *GoItab
Value unsafe.Pointer
}
type GoEface struct {
Type *GoType
Value unsafe.Pointer
}
func (self GoEface) Pack() (v interface{}) {
*(*GoEface)(unsafe.Pointer(&v)) = self
return
}
type GoPtrType struct {
GoType
Elem *GoType
}
type GoMapType struct {
GoType
Key *GoType
Elem *GoType
Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8
ElemSize uint8
BucketSize uint16
Flags uint32
}
func (self *GoMapType) IndirectElem() bool {
return self.Flags & 2 != 0
}
type GoStructType struct {
GoType
Pkg *byte
Fields []GoStructField
}
type GoStructField struct {
Name *byte
Type *GoType
OffEmbed uintptr
}
type GoInterfaceType struct {
GoType
PkgPath *byte
Methods []GoInterfaceMethod
}
type GoInterfaceMethod struct {
Name int32
Type int32
}
type GoSlice struct {
Ptr unsafe.Pointer
Len int
Cap int
}
type GoString struct {
Ptr unsafe.Pointer
Len int
}
func PtrElem(t *GoType) *GoType {
return (*GoPtrType)(unsafe.Pointer(t)).Elem
}
func MapType(t *GoType) *GoMapType {
return (*GoMapType)(unsafe.Pointer(t))
}
func IfaceType(t *GoType) *GoInterfaceType {
return (*GoInterfaceType)(unsafe.Pointer(t))
}
func UnpackType(t reflect.Type) *GoType {
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
}
func UnpackEface(v interface{}) GoEface {
return *(*GoEface)(unsafe.Pointer(&v))
}
func UnpackIface(v interface{}) GoIface {
return *(*GoIface)(unsafe.Pointer(&v))
}
func findReflectRtypeItab() *GoItab {
v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab
}

181
vendor/github.com/bytedance/sonic/loader/internal/rt/stackmap.go generated vendored Normal file
View File

@@ -0,0 +1,181 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package rt
import (
`fmt`
`strings`
`unsafe`
)
type Bitmap struct {
N int
B []byte
}
func (self *Bitmap) grow() {
if self.N >= len(self.B) * 8 {
self.B = append(self.B, 0)
}
}
func (self *Bitmap) mark(i int, bv int) {
if bv != 0 {
self.B[i / 8] |= 1 << (i % 8)
} else {
self.B[i / 8] &^= 1 << (i % 8)
}
}
func (self *Bitmap) Set(i int, bv int) {
if i >= self.N {
panic("bitmap: invalid bit position")
} else {
self.mark(i, bv)
}
}
func (self *Bitmap) Append(bv int) {
self.grow()
self.mark(self.N, bv)
self.N++
}
func (self *Bitmap) AppendMany(n int, bv int) {
for i := 0; i < n; i++ {
self.Append(bv)
}
}
// var (
// _stackMapLock = sync.Mutex{}
// _stackMapCache = make(map[*StackMap]struct{})
// )
type BitVec struct {
N uintptr
B unsafe.Pointer
}
func (self BitVec) Bit(i uintptr) byte {
return (*(*byte)(unsafe.Pointer(uintptr(self.B) + i / 8)) >> (i % 8)) & 1
}
func (self BitVec) String() string {
var i uintptr
var v []string
/* add each bit */
for i = 0; i < self.N; i++ {
v = append(v, fmt.Sprintf("%d", self.Bit(i)))
}
/* join them together */
return fmt.Sprintf(
"BitVec { %s }",
strings.Join(v, ", "),
)
}
type StackMap struct {
N int32
L int32
B [1]byte
}
// func (self *StackMap) add() {
// _stackMapLock.Lock()
// _stackMapCache[self] = struct{}{}
// _stackMapLock.Unlock()
// }
func (self *StackMap) Pin() uintptr {
// self.add()
return uintptr(unsafe.Pointer(self))
}
func (self *StackMap) Get(i int32) BitVec {
return BitVec {
N: uintptr(self.L),
B: unsafe.Pointer(uintptr(unsafe.Pointer(&self.B)) + uintptr(i * ((self.L + 7) >> 3))),
}
}
func (self *StackMap) String() string {
sb := strings.Builder{}
sb.WriteString("StackMap {")
/* dump every stack map */
for i := int32(0); i < self.N; i++ {
sb.WriteRune('\n')
sb.WriteString(" " + self.Get(i).String())
}
/* close the stackmap */
sb.WriteString("\n}")
return sb.String()
}
func (self *StackMap) MarshalBinary() ([]byte, error) {
size := int(self.N) * int(self.L) + int(unsafe.Sizeof(self.L)) + int(unsafe.Sizeof(self.N))
return BytesFrom(unsafe.Pointer(self), size, size), nil
}
var (
byteType = UnpackEface(byte(0)).Type
)
const (
_StackMapSize = unsafe.Sizeof(StackMap{})
)
//go:linkname mallocgc runtime.mallocgc
//goland:noinspection GoUnusedParameter
func mallocgc(nb uintptr, vt *GoType, zero bool) unsafe.Pointer
type StackMapBuilder struct {
b Bitmap
}
//go:nocheckptr
func (self *StackMapBuilder) Build() (p *StackMap) {
nb := len(self.b.B)
bm := mallocgc(_StackMapSize + uintptr(nb) - 1, byteType, false)
/* initialize as 1 bitmap of N bits */
p = (*StackMap)(bm)
p.N, p.L = 1, int32(self.b.N)
copy(BytesFrom(unsafe.Pointer(&p.B), nb, nb), self.b.B)
return
}
func (self *StackMapBuilder) AddField(ptr bool) {
if ptr {
self.b.Append(1)
} else {
self.b.Append(0)
}
}
func (self *StackMapBuilder) AddFields(n int, ptr bool) {
if ptr {
self.b.AppendMany(n, 1)
} else {
self.b.AppendMany(n, 0)
}
}

37
vendor/github.com/bytedance/sonic/loader/loader.go generated vendored Normal file
View File

@@ -0,0 +1,37 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`unsafe`
)
// Function is a function pointer
type Function unsafe.Pointer
// Options used to load a module
type Options struct {
// NoPreempt is used to disable async preemption for this module
NoPreempt bool
}
// Loader is a helper used to load a module simply
type Loader struct {
Name string // module name
File string // file name
Options
}

112
vendor/github.com/bytedance/sonic/loader/loader_latest.go generated vendored Normal file
View File

@@ -0,0 +1,112 @@
// +build go1.16,!go1.23
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`github.com/bytedance/sonic/loader/internal/rt`
)
// LoadFuncs loads only one function as module, and returns the function pointer
// - text: machine code
// - funcName: function name
// - frameSize: stack frame size.
// - argSize: argument total size (in bytes)
// - argPtrs: indicates if a slot (8 Bytes) of arguments memory stores pointer, from low to high
// - localPtrs: indicates if a slot (8 Bytes) of local variants memory stores pointer, from low to high
//
// WARN:
// - the function MUST has fixed SP offset equaling to this, otherwise it go.gentraceback will fail
// - the function MUST has only one stack map for all arguments and local variants
func (self Loader) LoadOne(text []byte, funcName string, frameSize int, argSize int, argPtrs []bool, localPtrs []bool) Function {
size := uint32(len(text))
fn := Func{
Name: funcName,
TextSize: size,
ArgsSize: int32(argSize),
}
// NOTICE: suppose the function has fixed SP offset equaling to frameSize, thus make only one pcsp pair
fn.Pcsp = &Pcdata{
{PC: size, Val: int32(frameSize)},
}
if self.NoPreempt {
fn.PcUnsafePoint = &Pcdata{
{PC: size, Val: PCDATA_UnsafePointUnsafe},
}
} else {
fn.PcUnsafePoint = &Pcdata{
{PC: size, Val: PCDATA_UnsafePointSafe},
}
}
// NOTICE: suppose the function has only one stack map at index 0
fn.PcStackMapIndex = &Pcdata{
{PC: size, Val: 0},
}
if argPtrs != nil {
args := rt.StackMapBuilder{}
for _, b := range argPtrs {
args.AddField(b)
}
fn.ArgsPointerMaps = args.Build()
}
if localPtrs != nil {
locals := rt .StackMapBuilder{}
for _, b := range localPtrs {
locals.AddField(b)
}
fn.LocalsPointerMaps = locals.Build()
}
out := Load(text, []Func{fn}, self.Name + funcName, []string{self.File})
return out[0]
}
// Load loads given machine codes and corresponding function information into go moduledata
// and returns runnable function pointer
// WARN: this API is experimental, use it carefully
func Load(text []byte, funcs []Func, modulename string, filenames []string) (out []Function) {
ids := make([]string, len(funcs))
for i, f := range funcs {
ids[i] = f.Name
}
// generate module data and allocate memory address
mod := makeModuledata(modulename, filenames, &funcs, text)
// verify and register the new module
moduledataverify1(mod)
registerModule(mod)
//
// encapsulate function address
out = make([]Function, len(funcs))
for i, s := range ids {
for _, f := range funcs {
if f.Name == s {
m := uintptr(mod.text + uintptr(f.EntryOff))
out[i] = Function(&m)
}
}
}
return
}

45
vendor/github.com/bytedance/sonic/loader/mmap_unix.go generated vendored Normal file
View File

@@ -0,0 +1,45 @@
//go:build !windows
// +build !windows
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`syscall`
)
const (
_AP = syscall.MAP_ANON | syscall.MAP_PRIVATE
_RX = syscall.PROT_READ | syscall.PROT_EXEC
_RW = syscall.PROT_READ | syscall.PROT_WRITE
)
func mmap(nb int) uintptr {
if m, _, e := syscall.RawSyscall6(syscall.SYS_MMAP, 0, uintptr(nb), _RW, _AP, 0, 0); e != 0 {
panic(e)
} else {
return m
}
}
func mprotect(p uintptr, nb int) {
if _, _, err := syscall.RawSyscall(syscall.SYS_MPROTECT, p, uintptr(nb), _RX); err != 0 {
panic(err)
}
}

84
vendor/github.com/bytedance/sonic/loader/mmap_windows.go generated vendored Normal file
View File

@@ -0,0 +1,84 @@
//go:build windows
// +build windows
// build
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`syscall`
`unsafe`
)
const (
MEM_COMMIT = 0x00001000
MEM_RESERVE = 0x00002000
)
var (
libKernel32 = syscall.NewLazyDLL("KERNEL32.DLL")
libKernel32_VirtualAlloc = libKernel32.NewProc("VirtualAlloc")
libKernel32_VirtualProtect = libKernel32.NewProc("VirtualProtect")
)
func mmap(nb int) uintptr {
addr, err := winapi_VirtualAlloc(0, nb, MEM_COMMIT|MEM_RESERVE, syscall.PAGE_READWRITE)
if err != nil {
panic(err)
}
return addr
}
func mprotect(p uintptr, nb int) (oldProtect int) {
err := winapi_VirtualProtect(p, nb, syscall.PAGE_EXECUTE_READ, &oldProtect)
if err != nil {
panic(err)
}
return
}
// winapi_VirtualAlloc allocate memory
// Doc: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc
func winapi_VirtualAlloc(lpAddr uintptr, dwSize int, flAllocationType int, flProtect int) (uintptr, error) {
r1, _, err := libKernel32_VirtualAlloc.Call(
lpAddr,
uintptr(dwSize),
uintptr(flAllocationType),
uintptr(flProtect),
)
if r1 == 0 {
return 0, err
}
return r1, nil
}
// winapi_VirtualProtect change memory protection
// Doc: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualprotect
func winapi_VirtualProtect(lpAddr uintptr, dwSize int, flNewProtect int, lpflOldProtect *int) error {
r1, _, err := libKernel32_VirtualProtect.Call(
lpAddr,
uintptr(dwSize),
uintptr(flNewProtect),
uintptr(unsafe.Pointer(lpflOldProtect)),
)
if r1 == 0 {
return err
}
return nil
}

93
vendor/github.com/bytedance/sonic/loader/pcdata.go generated vendored Normal file
View File

@@ -0,0 +1,93 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`encoding/binary`
)
const (
_N_PCDATA = 4
_PCDATA_UnsafePoint = 0
_PCDATA_StackMapIndex = 1
_PCDATA_InlTreeIndex = 2
_PCDATA_ArgLiveIndex = 3
_PCDATA_INVALID_OFFSET = 0
)
const (
// PCDATA_UnsafePoint values.
PCDATA_UnsafePointSafe = -1 // Safe for async preemption
PCDATA_UnsafePointUnsafe = -2 // Unsafe for async preemption
// PCDATA_Restart1(2) apply on a sequence of instructions, within
// which if an async preemption happens, we should back off the PC
// to the start of the sequence when resume.
// We need two so we can distinguish the start/end of the sequence
// in case that two sequences are next to each other.
PCDATA_Restart1 = -3
PCDATA_Restart2 = -4
// Like PCDATA_RestartAtEntry, but back to function entry if async
// preempted.
PCDATA_RestartAtEntry = -5
_PCDATA_START_VAL = -1
)
var emptyByte byte
// Pcvalue is the program count corresponding to the value Val
// WARN: we use relative value here (to function entry)
type Pcvalue struct {
PC uint32 // program count relative to function entry
Val int32 // value relative to the value in function entry
}
// Pcdata represents pc->value mapping table.
// WARN: we use ** [Pcdata[i].PC, Pcdata[i+1].PC) **
// as the range where the Pcdata[i].Val is effective.
type Pcdata []Pcvalue
// see https://docs.google.com/document/d/1lyPIbmsYbXnpNj57a261hgOYVpNRcgydurVQIyZOz_o/pub
func (self Pcdata) MarshalBinary() (data []byte, err error) {
// delta value always starts from -1
sv := int32(_PCDATA_START_VAL)
sp := uint32(0)
buf := make([]byte, binary.MaxVarintLen32)
for _, v := range self {
if v.PC < sp {
panic("PC must be in ascending order!")
}
dp := uint64(v.PC - sp)
dv := int64(v.Val - sv)
if dv == 0 || dp == 0 {
continue
}
n := binary.PutVarint(buf, dv)
data = append(data, buf[:n]...)
n2 := binary.PutUvarint(buf, dp)
data = append(data, buf[:n2]...)
sp = v.PC
sv = v.Val
}
// put 0 to indicate ends
data = append(data, 0)
return
}

60
vendor/github.com/bytedance/sonic/loader/stubs.go generated vendored Normal file
View File

@@ -0,0 +1,60 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
"sync/atomic"
"unsafe"
_ `unsafe`
)
//go:linkname lastmoduledatap runtime.lastmoduledatap
//goland:noinspection GoUnusedGlobalVariable
var lastmoduledatap *moduledata
func registerModule(mod *moduledata) {
registerModuleLockFree(&lastmoduledatap, mod)
}
//go:linkname moduledataverify1 runtime.moduledataverify1
func moduledataverify1(_ *moduledata)
func registerModuleLockFree(tail **moduledata, mod *moduledata) {
for {
oldTail := loadModule(tail)
if casModule(tail, oldTail, mod) {
storeModule(&oldTail.next, mod)
break
}
}
}
func loadModule(p **moduledata) *moduledata {
return (*moduledata)(atomic.LoadPointer((*unsafe.Pointer)(unsafe.Pointer(p))))
}
func storeModule(p **moduledata, value *moduledata) {
atomic.StorePointer((*unsafe.Pointer)(unsafe.Pointer(p)), unsafe.Pointer(value))
}
func casModule(p **moduledata, oldValue *moduledata, newValue *moduledata) bool {
return atomic.CompareAndSwapPointer(
(*unsafe.Pointer)(unsafe.Pointer(p)),
unsafe.Pointer(oldValue),
unsafe.Pointer(newValue),
)
}

185
vendor/github.com/bytedance/sonic/loader/wrapper.go generated vendored Normal file
View File

@@ -0,0 +1,185 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`reflect`
`unsafe`
`github.com/bytedance/sonic/loader/internal/abi`
`github.com/bytedance/sonic/loader/internal/rt`
)
var _C_Redzone = []bool{false, false, false, false}
// CFunc is a function information for C func
type CFunc struct {
// C function name
Name string
// entry pc relative to entire text segment
EntryOff uint32
// function text size in bytes
TextSize uint32
// maximum stack depth of the function
MaxStack uintptr
// PC->SP delta lists of the function
Pcsp [][2]uint32
}
// GoC is the wrapper for Go calls to C
type GoC struct {
// CName is the name of corresponding C function
CName string
// CEntry points out where to store the entry address of corresponding C function.
// It won't be set if nil
CEntry *uintptr
// GoFunc is the POINTER of corresponding go stub function.
// It is used to generate Go-C ABI conversion wrapper and receive the wrapper's address
// eg. &func(a int, b int) int
// FOR
// int add(int a, int b)
// It won't be set if nil
GoFunc interface{}
}
// WrapGoC wraps C functions and loader it into Go stubs
func WrapGoC(text []byte, natives []CFunc, stubs []GoC, modulename string, filename string) {
funcs := make([]Func, len(natives))
// register C funcs
for i, f := range natives {
fn := Func{
Flag: FuncFlag_ASM,
EntryOff: f.EntryOff,
TextSize: f.TextSize,
Name: f.Name,
}
if len(f.Pcsp) != 0 {
fn.Pcsp = (*Pcdata)(unsafe.Pointer(&natives[i].Pcsp))
}
// NOTICE: always forbid async preempt
fn.PcUnsafePoint = &Pcdata{
{PC: f.TextSize, Val: PCDATA_UnsafePointUnsafe},
}
// NOTICE: always refer to first file
fn.Pcfile = &Pcdata{
{PC: f.TextSize, Val: 0},
}
// NOTICE: always refer to first line
fn.Pcline = &Pcdata{
{PC: f.TextSize, Val: 1},
}
// NOTICE: copystack need locals stackmap
fn.PcStackMapIndex = &Pcdata{
{PC: f.TextSize, Val: 0},
}
sm := rt.StackMapBuilder{}
sm.AddField(false)
fn.ArgsPointerMaps = sm.Build()
fn.LocalsPointerMaps = sm.Build()
funcs[i] = fn
}
rets := Load(text, funcs, modulename, []string{filename})
// got absolute entry address
native_entry := **(**uintptr)(unsafe.Pointer(&rets[0]))
// println("native_entry: ", native_entry)
wraps := make([]Func, 0, len(stubs))
wrapIds := make([]int, 0, len(stubs))
code := make([]byte, 0, len(wraps))
entryOff := uint32(0)
// register go wrappers
for i := range stubs {
for j := range natives {
if stubs[i].CName != natives[j].Name {
continue
}
// calculate corresponding C entry
pc := uintptr(native_entry + uintptr(natives[j].EntryOff))
if stubs[i].CEntry != nil {
*stubs[i].CEntry = pc
}
// no need to generate wrapper, continue next
if stubs[i].GoFunc == nil {
continue
}
// assemble wrapper codes
layout := abi.NewFunctionLayout(reflect.TypeOf(stubs[i].GoFunc).Elem())
frame := abi.NewFrame(&layout, _C_Redzone, true)
tcode := abi.CallC(pc, frame, natives[j].MaxStack)
code = append(code, tcode...)
size := uint32(len(tcode))
fn := Func{
Flag: FuncFlag_ASM,
ArgsSize: int32(layout.ArgSize()),
EntryOff: entryOff,
TextSize: size,
Name: stubs[i].CName + "_go",
}
// add check-stack and grow-stack texts' pcsp
fn.Pcsp = &Pcdata{
{PC: uint32(frame.StackCheckTextSize()), Val: 0},
{PC: size - uint32(frame.GrowStackTextSize()), Val: int32(frame.Size())},
{PC: size, Val: 0},
}
// NOTICE: always refer to first file
fn.Pcfile = &Pcdata{
{PC: size, Val: 0},
}
// NOTICE: always refer to first line
fn.Pcline = &Pcdata{
{PC: size, Val: 1},
}
// NOTICE: always forbid async preempt
fn.PcUnsafePoint = &Pcdata{
{PC: size, Val: PCDATA_UnsafePointUnsafe},
}
// register pointer stackmaps
fn.PcStackMapIndex = &Pcdata{
{PC: size, Val: 0},
}
fn.ArgsPointerMaps = frame.ArgPtrs()
fn.LocalsPointerMaps = frame.LocalPtrs()
entryOff += size
wraps = append(wraps, fn)
wrapIds = append(wrapIds, i)
}
}
gofuncs := Load(code, wraps, modulename+"/go", []string{filename+".go"})
// set go func value
for i := range gofuncs {
idx := wrapIds[i]
w := rt.UnpackEface(stubs[idx].GoFunc)
*(*Function)(w.Value) = gofuncs[i]
}
}