Documentation
¶
Index ¶
- func AddFileToDirectory(root *Directory, filePath string)
- func BasicEstimateTokenFromChars(charCount int) int
- func CountFilesRecursive(dir *Directory) int
- func GenerateDotOutput(root *Directory) string
- func GenerateJSONTreeOutput(root *Directory) string
- func GenerateTreeOutput(root *Directory) string
- func GenerateXMLTreeOutput(root *Directory) string
- type Directory
- type TokenEstimator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddFileToDirectory ¶
AddFileToDirectory adds a file to the directory structure
func BasicEstimateTokenFromChars ¶
BasicEstimateTokenFromChars provides a simple estimate of tokens based on characters
func CountFilesRecursive ¶
CountFilesRecursive counts the number of files in a directory and its subdirectories
func GenerateDotOutput ¶
GenerateDotOutput generates a DOT (Graphviz) representation of the directory tree
func GenerateJSONTreeOutput ¶
GenerateJSONTreeOutput generates a JSON representation of the directory tree
func GenerateTreeOutput ¶
GenerateTreeOutput generates a string representation of the directory tree
func GenerateXMLTreeOutput ¶
GenerateXMLTreeOutput generates an XML representation of the directory tree
Types ¶
type Directory ¶
Directory represents a directory in the file system
func GetOrCreateDirectory ¶
GetOrCreateDirectory gets or creates a directory at the specified path
func NewDirectory ¶
NewDirectory creates a new directory structure
type TokenEstimator ¶
type TokenEstimator struct {
// Adjustment factors for different categories of text
CodeFactor float64
TextFactor float64
SpecialChars map[rune]float64 // Special characters and their token weight
}
TokenEstimator estimates token counts from text
func NewTokenEstimator ¶
func NewTokenEstimator() *TokenEstimator
NewTokenEstimator creates a new token estimator with default settings
func (*TokenEstimator) EstimateFileTokens ¶
func (te *TokenEstimator) EstimateFileTokens(filePath string, isCode bool) (int, error)
EstimateFileTokens estimates the number of tokens in a file
func (*TokenEstimator) EstimateTokens ¶
func (te *TokenEstimator) EstimateTokens(text string, isCode bool) int
EstimateTokens estimates the number of tokens in the given text The isCode parameter indicates whether the text is code or natural language
Source Files