• ☆ Yσɠƚԋσʂ ☆@lemmygrad.mlOP
    link
    fedilink
    arrow-up
    3
    arrow-down
    1
    ·
    16 days ago

    Not sure if anybody is hosting it online aside from the official instance. Running locally or using something like GCP to host your own private model is basically the way to go. Unfortunately, smaller sized models you can run locally aren’t quite as good as the full blown version. Still can be quite useful though, and they are rapidly improving as well. I’ve found qwen 3 to be the best local option right now.

    • darkernations@lemmygrad.ml
      link
      fedilink
      arrow-up
      2
      ·
      edit-2
      16 days ago

      Thanks for explaining. Is it easy to turn off censorship in qwen 3 (“non-coder” here but that doesn’t matter as I will be starting that journey soon enough as I up my maths)? Also I thought the free version of deepseek r1 on the app is 7b?

      • ☆ Yσɠƚԋσʂ ☆@lemmygrad.mlOP
        link
        fedilink
        arrow-up
        2
        arrow-down
        1
        ·
        16 days ago

        You don’t have to do anything, the open source models that are published don’t censor themselves. And the online version of deepseek is the full model. They charge for API level access for it.

        • CriticalResist8@lemmygrad.ml
          link
          fedilink
          arrow-up
          1
          ·
          15 days ago

          We are playing around with the idea on ProleWiki of automatically translating our pages to our different language instances, instead of having to do it ourselves which we’re not doing in the first place lol. Would you possibly have any ideas for that? There’s no limit to how much we could automate the process with API access, but I’m also wondering if we can do it cheaply (considering our funds) and which AI would be best for this. Might even want to look at running our own AI on someone’s machine. Anyway, I’m taking any suggestion lol

          • ☆ Yσɠƚԋσʂ ☆@lemmygrad.mlOP
            link
            fedilink
            arrow-up
            2
            arrow-down
            1
            ·
            15 days ago

            I’ve found DeepSeek works pretty well for translating content, their API access is pretty cheap. The main limitation comes from the context size, smaller models can handle less text, so you’d have to feed it content in smaller chunks. That said, locally running models are pretty capable of doing these types of translations.

            Here’s an example node script you could use to call DeepSeek to translate a document:

            const fs = require('fs');
            const axios = require('axios');
            
            async function translateFile(filePath) {
                try {
                    // Check if API key is set
                    if (!process.env.DEEPSEEK_API_KEY) {
                        throw new Error('DEEPSEEK_API_KEY environment variable is not set');
                    }
            
                    // Read the file content
                    const content = fs.readFileSync(filePath, 'utf8');
                    
                    // Call Deepseek API for translation using chat completion
                    const response = await axios.post(
                        'https://api.deepseek.com/v1/chat/completions',
                        {
                            model: "deepseek-chat",
                            messages: [
                                {
                                    role: "system",
                                    content: "You are a professional translator. Translate the following text to English while preserving formatting and meaning."
                                },
                                {
                                    role: "user",
                                    content: content
                                }
                            ],
                            temperature: 0.3
                        },
                        {
                            headers: {
                                'Authorization': `Bearer ${process.env.DEEPSEEK_API_KEY}`,
                                'Content-Type': 'application/json'
                            }
                        }
                    );
            
                    return response.data.choices[0].message.content;
                } catch (error) {
                    console.error('Translation failed:', error.message);
                    process.exit(1);
                }
            }
            
            // Usage: node src/index.js <input-file> <output-file>
            async function main() {
                if (process.argv.length < 4) {
                    console.log('Usage: node src/index.js <input-file> <output-file>');
                    process.exit(1);
                }
            
                const inputFile = process.argv[2];
                const outputFile = process.argv[3];
                const translatedText = await translateFile(inputFile);
                
                fs.writeFileSync(outputFile, translatedText);
                console.log(`Translation saved to ${outputFile}`);
            }
            
            main();
            

            Similarly, if you wanted to use a model like qwen3 with ollama for translations, you could do something like this:

            const fs = require('fs');
            const axios = require('axios');
            
            async function translateText(text) {
                try {
                    const response = await axios.post('http://localhost:11434/api/generate', {
                        model: 'qwen3:32b',
                        prompt: `Translate the following text to English:\n\n${text}`,
                        stream: false,
                        options: {
                            num_ctx: 16384,  // Larger context window
                            temperature: 0.3,  // More deterministic output
                            top_k: 40,       // Balance between quality and speed
                            top_p: 0.9        // Controls diversity of output
                        }
                    });
                    return response.data.response;
                } catch (error) {
                    console.error('Translation error:', error);
                    throw error;
                }
            }
            
            async function translateFile(inputPath, outputPath) {
                try {
                    // Read input file
                    const inputText = fs.readFileSync(inputPath, 'utf8');
                    
                    // Translate text
                    const translatedText = await translateText(inputText);
                    
                    // Write output file
                    fs.writeFileSync(outputPath, translatedText);
                    console.log(`Translation complete. Output written to ${outputPath}`);
                } catch (error) {
                    console.error('File processing error:', error);
                }
            }
            
            // Usage: node src/index.js input.txt output.txt
            if (process.argv.length >= 4) {
                const inputFile = process.argv[2];
                const outputFile = process.argv[3];
                translateFile(inputFile, outputFile);
            } else {
                console.log('Usage: node src/index.js <input-file> <output-file>');
            }