I have a decent CPU and GPU with 12GB VRam - this should let me run the 7B at least, from what I have seen in the sticky post.

Beside downloading the model, what kind of UI should I start with? Are there good tutorials around, that you are aware of?

Thanks for the input!

I recently built a new PC to handle Stable Diffusion, that gives me 12GB of VRAM to work with. I also started to self-host a few things on a VPS recently, so I have a bit of a basis there.

As for Stable Diffusion integration: I do storytelling/worldbuilding as a hobby and find LLM’s to be an amazing tool to “brainstorm with myself”. It would be amazing if I could tell the LLM to “make a picture of the new character” and it would connect to and prompt SD accordingly. I assume that this is out of scope of what’s currently possible, but something like that would be my goal. I will certainly have a look at langchain as you proposed. Also that’s the context of me asking about cross referencing and context length. I’ve been working with ChatGPT, and while it is an amazing tool it had me bummed out when it couldn’t reference a character that was developed a couple hours earlier (even in the same thread). The cross referencing solution that you sketched above might work for me, but I guess it’ll take a while to learn how to do it.

Given this as a bit of context: where should I start? Downloading Llama 2 as another reply suggests and go from there?

Yes, the final setup would be to run Adguard on a docker container and have this container be in a VPN. I’m not sure yet, how I would do that without messing up the other things already running on that VPS. Maybe I will go for Raspberry to run adguard at home.

Edit: I have set the second DNS in Windows to the same IP, so it shouldn’t have a fallback.

Edit: I fixed my problem by re-making my nginx reverse proxy and a do-over of my proxy hosts. I have yet to restart my server, though …

I’m a beginner with all of this stuff, so I’m sure I’m not assessing correctly what’s wrong with my setup. It’s more of a methodical “trial and error” approach, that I have, where I change one thing at a time and see what happens … quite time consuming but it helps me to figure things out along the way :)

However, if you have an idea, what could be wrong with my server, I’d appreciate any ideas: I run Nginx Reverse Proxy with nginx in a container within a custom network “my_network” and have assigned that container a fixed IP. I run other containers (portainer, mariadb, nextcloud, synapse) that all connect to the same custom network. The nginx container “see’s” the outside web with ports 80 and 443 openend on the firewall for that container’s fixed ip and routes traffic (and needed other ports) to my other containers. This is all working well and also works after restarting the server.

Now I tried to install a lemmy instance and got it up and running by bringing the lemmy containers in my custom network as well and proxy’img my nginx to the lemmy proxy. However, when I made a restart of the server, something broke and I cannot get the web-ui of NPM to load. I think somehow host names and/or IP adresses got mixed up somewhere. The containers start just fine, but I can’t access it with web-ui anymore. Also reverse proxy-ing doesn’t work, but if I open the needed ports on my firewall manually I can access the other services containers.

I hope this is even understandable, not sure if I’m using the correct terms …

thanky you, this looks like exactly what I need.

I do run several webservices (nextcloud, matrix) behind the same reverse proxy (nginx prxy manager). In my setup I have one docker with nginx running, which is the only one to be exposed to the web. It proxy-ing for the other services relies upon them being in the same network. It all works well, however I ran into problems when restarting my server after a shutdown. I suspect that some of the services tried to get the same ip adress as my nginx service, which results in that service not running properly and my whole reverse proxy setup falls apart at that point.

I’m not certain, that this is really what happens but I want to try and assign the fixed ip’s and see if that solves the problem.

Which domain name should I put in the nginx configuration from Lemmy? My intended domain (like lemmy.my-domain.tld) or do I put some internal IP (e.g. 172.20.0.1) and point to that IP from my host nginx?

In the configuration of the docker proxxy, do I define my domain name (like lemmy.my-domain.tld) or will I define some local IP (like 172.20.0.1) and let nginx proxy manager point to that?

thank you, that clears things up a bit. Now it’s to play around with it, until I get it up and running :)

Could you have a look at my answer to the poster above - would multiplexing mean, that I configure my internal IP 0.0.0.0:XXXA for one service and 0.0.0.0:XXXB for another?

This makes it clearer to my, would you mind helping me to understand all steps for my usecase. I want to run a lemmy instance and a mastodon instance on the same VPS, using the same domain but different subdomains - lmy.my-domain.tld and mstdn.my-domain.tld. I have my VPS IP address and setup the 2 subdomains with my domain provider (both subdomains are resolving the same IP).

I also did setup nginx on my server and can install SSL certificates for both of these domains. I’m now at the step where lmy.my-domain.tld should by directed to the lemmy service and mstdn.my-domain.tld to the mastodon service. As I understand it, both services listen to the ports 80 (http) and 443 (https). Do I now setup a room/building for Lemmy / Mastodon respectively where I tell nginx that lmy.my-domain.tld is at 0.0.0.0:3001 and mstdn.my-domain.tld is at 0.0.0.0:3002 for example. And in the config files for each of these installs I’d specify “0.0.0.0:300x” respectivly? (also have to make sure, that these docker installs don’t mess with my nginx config by themselves, right?)

that seems to have been part of the problem, as I indeed had nginx running on the host as well. Now I get the error code “website cannot be reached” when I try to go to my instance in the browser.

I tried to follow the configuration for nginx as was in the template file on github, but I most probably have an error there. One thing confuses me, that’s the ports for lemmy and the lemmy UI. I think they should be 8536 an 1235 respectively, but sometimes it says 1234 and 1236 for the UI port as well. Also in the template I’m using (https://github.com/LemmyNet/lemmy-ansible/blob/main/templates/nginx.conf#L63) there is only one section to enter ports: proxy_pass http://0.0.0.0:{{lemmy_port}}; - which port do I enter here?

if you happen to know, please let me know :)

Something seems to have broken specifically when using Firefox. Trying to open https://lemmy.world results in “404 - deleted” in my case. I can open lemmy.world just fine on Safari (iOS) and Edge (Win11).

I just checked out YUNoHost, which sounds like an excellent stepping stone, thanks for the recommendation! I’ll go with a VPS, so ISP shouldn’t be a concern. I think, I’ll just try it out - at worst I’ll lose a couple of hours of my time ;)

It’s the prime reason, I didn’t get D4 yet. I don’t trust the “it’s all only cosmetic” spin, they are drumming up. Battle Passes exist to make money, they don’t exist to make gamers happy. So, I’ll wait and see until we exactly know what’s in the battle pass, and how obnoxious they advertise it and what kind of nudges there are in place to fork over money. If it turns out to be something that indeed can be ignored, I will probably jump into D4. If the battle pass cannot be ignored, I’ll stay away.